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PREFACE 

This  report  is  an  augmented  version  of  a report  originally  Issued  in  September  of 
1976,  during  the  demonstration  ai  the  end  Of  the  five-year  speech  effort.  The  first 
section  reports  on  the  various  speech  understanding  syslems  developed  at  CMU  during 
the  five  year  period  and  highlights  their  indn'idjal  contributions.  Section  II  contains  a 
brief  description  of  several  techniques  and  Knowledge  sources  that  contributed  to  the 
success  of  the  final  systems.  Section  111  gives  detailed  performance  results  of  the 
Harpy  and  Hearsay-II  systems.  Results  include  the  performance  of  the  systems  not 
only  for  the  1000  word  task  but  for  several  simpler  tasks.  Section  IV  contains 
reprints  of  papers  presented  at  various  conferences  since  September  1976.  Section 
V LOfitains  a list  of  publications  of  the  CMU  speech  group. 
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1.  MULTI-SYSTEM  APPROACH  TO  SPEECH  UNDERSTANDING* 

Raj  Re^idy 


INTRODUCTION 


In  1971,  a group  of  srienhsts  recommer-.dod  the  initiation  o*  a five-year  research 
program  towards  the  demonstration  of  a large-vocabulary  connected  speech 
understanding  system  (Newell  et  at.,  1971).  instead  o*  setting  vague  oojcctives,  the 
group  proposed  a set  of  scecific  performance  goals  (see  Fig.  1.'  of  Newell  et  al.. 
1971).  The  system  was  required  to  accept  connected  speech  from  many  speakers 
based  On  a 1000  word  vocabulary  task-oriented  grammar,  within  a constrained  task. 
The  system  was  expected  to  perform  with  le  s than  107-  semantic  errors,  usir'c  about 
300  million  instructions  per  second  of  speech  (MiPSS)**  and  to  be  operational  within  a 
five  year  period.  The  proposed  research  wa-,  a highly  ambitious  undertaking,  given 
the  almost  total  lack  of  experience  with  connected  speech  systems  at  that  time. 


The  Harpy  and  Hearsay-ll  systems  dev' 
the  best  overall  performance  at  the  end  of  t 
the  performance  of  the  Harpy  system,  relati 
only  satisfies  the  original  goals,  but  exce’ 
recogniaes  speech  from  male  and  female  ; 
document  retrieval  task.  Semantic  error  is  5 
faster  th.an  expected  The  Hearsa/-ll  system 
2 to  20  tirr.es  slower  than  Harpy. 

Of  the  many  factors  that  lea  to  the 
systems,  perhaps  the  i-CiOst  important  was  th. 
evolved.  Faced  with  prospects  of  develo. 
Linknowns,  we  Opted  to  develop  ses'eral  m’ 
than  wc^rK  towards  a single  carefully  design 
these  intermediate  systems  were  deliberately 
understanding  of  some  aspect  of  the  overall 

GOAL  (Nov.  1971; 

Accept  connected  speech 
from  many 

cooperative  speakers 
m a quiet  room 
using  a good  microphone 
witti  slight  tuning/speaker 
accepting  1000  words 
using  an  artificial  synla.i 
m a constraining  task 
yielding  107  semantic  error 
requir.ng  approx.  300  MiPSb** 


doped  at  Carnegie-Mtllon  University  had 
ne  five  year  period.  Figure  1 illustrates 
1 to  the  original  specifications.  It  not 
(is  som.e  of  the  stated  objectives.  It 
oeakers  using  a 101 1 -word-vocabulary 
' and  response  is  an  order  of  magnitude 
achieves  similar  accuracy  and  runs  about 

final  successful  demonstration  of  these 
systems  development  methodology  that 
ling  systems  with  a large  nurvtber  of 
■■'r■•cd!ate  "throw-away*  systems  'ather 
■'i  ulti.mate  system.  Many  dimensions  of 
imessed  or  ignored  so  as  to  gam  deeper 
/Stem  The  purpose  of  this  paper  is  to 

harpy  (iMov.  1976) 

Yes 

5 (3  male,  2 female) 
yes 

computer  term.ina!  room 
dose-talking  microphone 
20-30  senlences/talker 
iOl  i word  vocabulary 
a\;,  branching  factor  = 33 
cioc'jment  retrieval 
5.'- 

rcqi.iring  28  MIP5S 

us  rg  256k  of  36  bit  words 

costing  S5  per  sentence  processed 


Figure  1.  Harpy  performance  cun-.pa.'cd  to  desired  goals 

* ’aper  to  apoear  m Carnegie-Meilon  Comout  ’ Sc.erice  Researcti  Review,  1977 
**  The  actual  specifications  staled  "a  few  l.mes  rea'.-i'me"  on  a 100  MIPS  (Million 
I r.c,  I r i,ir  t lOn  ; n^r  rprnnri}  marhini? 


TasK  characteristics 

speakers;  number,  male/female,  dialect 
vocabulary  and  synlai 
response  desired 

Signal  gathering  environment 
room  noise  level 
transducer  characteristics 

Signal  trarrsformations 

digitiaahon  speed  and  accuracy 
special-purpose  hardware  required 
parametric  representation 

Signal-to-symbol  transformation 
segmentation’ 
level  transformation  occurs 
label  selection  technique 
arnount  of  training  required 

Matching  and  searching 

relaxation;  breadth-first 

• blackboard;  best-first,  island  driven 

productions;  best-first 
Locus;  beam  search 

Knowledge  source  representation 
networks 
procedures 
frames 
productions 

System  organization 

levels  of  representation 

Single  processor  / multi-processor 

Figure  2.  Design  choices  for  speech  understanding  systems. 


illustrate  the  incremental  understanding  of  the  solution  space  provided  by  the  various 
intermediate  systems  developed  at  CMU 

Figure  2 illustrates  the  large  number  of  design  decisions  which  confront  a 
speech  understanding  system  designer*.  For  each  of  these  10  to  15  design  decisions, 
we  have  3 to  10  feasible  alternative  choices.  Thus  the  solution  space  for  speech 
systems  seems  to  contain  10®  to  10®  possible  system  designs.  Given  the  interactions 
oetween  design  choices,  it  is  not  possible  to  evaluate  each  design  choice  in  isolation 
outside  the  framework  of  the  total  system. 

* Further  discussion  of  many  of  these  design  choices  can  be  found  in  RecJdy 
(1976). 
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jj.i,  <.... 


SYSTEMS 


f-'igure  3 show<i  the  genealogy  of  fhe  |.oeth,  understanding  systems  developed 
at  CMt^i  In  this  section  will  briefly  oulliar  Hic  u'.tcresting  aspects  Of  each  of  these 
systems  and  discuss  their  contributions  towards  tfie  development  of  speech 
understanding  sy'’cms  tectmology.  More  complete  descriptions  ot  these  systems  can 
be  found  m the  references  listed  at  *he  end. 


The  Hear  say- 1 System  (Erman,  Fennell,  Lowerre,  Neely,  and  Reddy)* 

Hearsay-1  (Reddy,  Erma-,  and  Neely  Reddy,  Erman,  Fennell  and  I'Jee', 

1973},  the  first  speech  understanding  system  developed  at  Car negie-Mellon  University, 
was  demofistr aled  m June  Of  1972.  This  syst-?m  was  one  of  the  first  connected  speech 
understanding  systems  to  use  task  dependerJ  knowiedge  to  achiC  'e  reduction  ot  the 
search  space  Recognition  uses  a best-first  s>:arcfi  strategy. 

Model 

Hearsay-1  was  the  first  system  to  utilize  independent,  cooperaling  knowledge 
sources  and  the  concept  of  a global  data  l:>.isc,  or  'blackboard',  through  which  all 
knowledge  sources  communicate.  Knowledge  'Ources  consist  of  the  acoustic -phonetic, 
syntactic,  and  semantic  modules.  Each  moouh’  operates  m the  "hypothesize-and-test" 
mode.  Synch.- cnous  activation  of  ‘l»e  modult  s leads  to  a best -first  search  strategy. 
Several  other  systems  have  useo  this  strateg.  (Foigic  1974).  This  system  was  one  of 
the  first  to  use  syntactically  derived  word  d ap.i  ams  and  tr'grams,  as  anti-productions 
'^l^ely  1973),  to  predict  forv  ard  and  back  ward  from  “islands  of  reliability".  Task 
dependent  Kncwledge,  such  ?s  a board  po'  hon  m die  chess  task,  is  used  by  the 
semantic  module  (Neely  1973)  to  reject  neanu. glass  partial  parses  early  in  the 
recognition  process  The  acoiistic-phonelic  i^.ociulc  uses  amplitude  and  zero-crossing 
parameters  to  obtain  a multilevel  segmentai  on  mlo  syilable-size  and  phoneme-size 
units  (Erman,  1974). 

Pertorrriance 

Over  a wide  range  of  tasks,  the  averaj  e scnlct'.cc  error  rate  was  69  7 M ith  a a 
word  error  rate  ot  457.  speed  varied  betwi . n 3 and  15  M1P5S  over  162  utterances 
coiitaining  57S  words  Mearsay-I  yields  mucr  ingher  accurac  es  on  tasks  with  which  it 
IS  carefully  trained  for  the  chess  task,  for  ir  -lance,  a.erage  seritencc  and  word  error 
rates  were  21  and  7 percent,  respectively,  ‘viii;  an  average  speed  of  2 K''!PSS. 

Oiscussion 

Hearsay-!,  as  a successful  connected-  pc  ech  understanding  system,  served  to 
Clarify  the  nature  and  necessary  mleractiOi'  cf  se-eral  sources  of  knowledge.  Its 
f'exibilily  provided  a means  for  leslmg  and  e .ilualmg  competing  theories,  allowing  the 
belter  thrones  to  be  chosen  as  a basis  for  ■air'-  systems  In  retrospect,  we  believe 
this  SYstere  organization  would  have  been  aoi'quate  tor  the  ARPA  specifications  given 
preserit  acou ctic-phonetic  Knowledge 

* The  principle  contributors  towards  Ihe  de  ■ .opment  of  each  of  these  systems  are 
listed  wittun  parentheses. 


The  Dragon  System  (Baker) 

Baker  (orfnuSated  the  recognition  proc''C'-  as  a Oynamic  pt  ogramirting  probtem. 
The  Dragon  recognition  system  (Baker,  n7S),  based  on  this  model  was  first 

dt n^onstrated  m April  of  1974.  The  svstei  . *a  n.otivated  by  a des.'C  to  use  a 

general  abstract  rnodel  to  represent  knoviecige  sources  The  model,  that  of  a 

probabilistic  function  of  a Markov  process,  <5  iTctiblc  and  leads  to  features  which  allow 

it  to  function  (despite  high  error  rates.  Recofniton  accuracy  was  greater  with  Dragon 
than  with  Mearsay-I,  but  the  system  ran  signit 'c  anily  s'ower. 

Model 

Dragon  was  the  first  system  to  demon‘'trate  the  use  of  z Markov  model  and 
dynamic  pi  ogrammir.g  :n  a connected  speech  ufvdcrstanding  system.  It  included  several 
interesting  features,  such  as  delayed  deci5iC’'s  and  integrated  representation,  and  is 
based  on  a gerseral  theorehcat  framework.  The  general  framework  allows  acoustic - 
phonetic,  syntactic,  and  semantic  knowledge  o be  embodied  in  a finite-state  network. 
E!ach  path  through  this  precompiled  network  represents  an  allowed  pronunciation  of  a 
syntactically  acceptable  sentence  Recognil.on  proceeds  left -to -right  through  the 
network,  searching  all  possible  paths  in  paral'ni  tc  determine  the  globally  optimal  path 
(i.e.,  the  path  which  best  matches  the  spoken  utterance).  Acoustic  inputs  are  peak-to- 
peak  amplitudes  and  zero-crossmgs  trorn  ovei  la.'omg,  one-third  octave  filters,  sampled 
every  centi-second 

PerformarKe 

Recognition  accuracy  was  greater  with  Dr.-gon  tnan  that  obtained  with  Hearsay- 
i,  but  at  a cost  of  speed.  Dragon  being  app  ou.malcl/  5 to  10  tinies  slower.  O.-er  a 
Wide  variety  of  Iasi's,  the  average  sentence  error  rale  was  517.  Speed  ranged  from 
14  to  50  MIPSS  The  computation  is  essentoiiy  linear  wth  !he  number  of  states  in  the 
Markov  network.  Performance  was  later  impr-ived  by  LOwerre  (Lowerre,  1976). 

Discussion 

Dragon,  with  more  accurate  performa  'cr  than  Hearsay'!,  served  to  stimulate 
further  research  into  factors  that  led  to  its  n proved  performance  Many  of  the  ideas 
motivat'ng  its  design  were  important  in  tne  oevo  opment  of  suosequent  conriected- 
speech  understanding  systems.  Although  lali  r s 'stems  do  riot  use  the  Markov  Model 
and  do  not  guarantee  finding  the  globally  i.pt.mal  path,  the  concepts  of  integrated 
representation  of  knowledge  sources  and  delayed  decisions  proved  to  be  very 
val'uable. 


The  Harpy  System  (Lowerre  and  Rei^y) 

The  Marpy  system  (Lowerre  1976)  was  the  first  connected  speech  system  to 
satisfy  the  original  spec  ific  at'ons  given  the  ^Towell  report  and  was  first 
demonstrated  m September  of  1976.  System  design  'was  motivated  by  an  investigation 
of  the  important  design  cnoices  contnbuti  ■ , to  t' e success  of  the  Dragon  and 
tivarsay-1  syslems  The  result  was  a ccrrioin  ,cn  of  Ine  “best“  features  of  these  two 
systems  with  additional  heuristics  to  g've  high  peed  and  accuracy. 

Model 

The  Harp-y  system  uses  the  locus  modfi  o’  locus  model  of  search,  a 

very  successful  search  technique  m speecn  understanding  rpsea'ch,  is  a graph- 
searching  technique  in  *ihich  all  eveept  a b(  .■'ni  of  riear-miss  a'ternatives  around  the 


bp't  path  are  pruned  frorri  the  search  tree  at  each  segmental  decision  point,  thus 
containing  the  exponential  growth  without  reci'iiring  backtracKing.  This  technique-  was 
instrumental  in  making  Harpy  the  most  successful  connected  speech  understanding 
system  to  date.  Harpy  represents  syntactic,  lexical,  and  juncture  K.ncwledge  in  a 
unified  ne.wQrk  as  in  Dragon,  but  without  the  a-priori  transition  probabilities.  Phonetic 
classification  is  accompl’shed  by  a set  <’i  cnpaKer-dependent  acoustic-phonetic 
templates  based  on  LPC  parameters  which  represent  the  acoustic  realizations  of  the 
phones  m the  lexical  portion  of  the  network. 

Performance 

The  system  was  tested  on  several  diHerent  tasks  with  biiierent  vocabularies 
and  branching  factors.  On  the  lOll-woni  task  using  the  AIX05  grammar  (see 
Appendix  Ill-C),  the  system  word  error  rate  was  37  and  the  semantic  error  rate  was 
5/  (see  fig.  1).  The  system  was  also  tested  v.'ith  connected  digits  recognition  attaining 
a 27  word  error  rate.  Using  speaker-indept ndent  templates,  error  rate  increases  to 
77-  Over  20  speaker  including  10  new  speak'^rs.  Using  telephone  input  increases  the 
error  rate  to  77,  to  117  depending  on  the  noise  characteristics  of  the  telephone 
systep'. 

Discussion 

' Backtracking  and  redundant  compulation  have  always  been  problematic  in  AI 
systems.  The  Harpy  system  eliminates  these  m an  elegant  way,  using  the  beam  search 
technique  By  compiling  knowledge  ahead  of  time,  Harpy  achieves  a level  of  efficiency 
that  is  unattainable  by  systems  that  dynaiucally  interpret  their  knowledge.  This 
permits  Harpy  to  consider  many  more  alternatives  and  deal  with  error  and  uncertainty 
m a graceful  manner. 


The  Hearsay-ll  System  (Erman,  Hayes-Foth,  Lesser,  and  Peddy) 

Hearsay-Il  has  been  the  major  researrh  effort  of  the  CMU  speech  group  Over 
the  last  three  years.  During  this  period,  solutions  were  devised  to  many  difficult 
conceptual  problems  that  arose  during  the  implementation  of  Hearsay-I  and  other 
earlier  efforts.  The  result  represents  not  onl/  ar.  mleresting  system  design  tor  speech 
unde,  st  andiiig  but  also  an  experiment  in  the  area  of  knowledge-based  systems 
architecture.  Attempts  are  being  made  by  cthei  Al  groups  to  use  this  type  of 
architecture  m image  processing  and  other  knnwipcige-intensive  systems. 

Hearsay-lI  is  similar  to  Hearsay-1  in  that  it  is  based  on  the  hypothesize-and-test 
paradigm,  using  cooperating  independent  kno.*. ledge  sources  communicating  through  a 
global  data  structure  (o' ’cl'board).  It  differs  m tiie  scr.se  that  many  of  the  limitations 
and  shorlcomings  of  Hearsay-i  are  resolved  m Hparsay-Il. 

Hearsay-!'  differs  from  the  Harpy  syst.  m m that  it  views  knowledge  sources  as 
diifereni  and  independent  and  thus  cannr.!  always  be  integrated  into  a single 
representation,  '^urther,  it  has  as  a design  v.oa!  the  ability  to  recognize,  understand, 
and  respond  even  in  situations  "here  senteiiccs  cannot  be  guaranleed  to  agree  with 
some  predefined,  restricted  language  model  a-,  is  the  case  with  the  Harpy  system. 

Model 

The  mam  features  of  the  Hearsay-II  sy  Icin  structure  arei  1)  the  representation 
of  knowledge  as  self  activating,  async  ironous.  parallel  processes,  2)  the 
representation  of  the  partial  analysis  m a generalized  three-dimensional  network;  the 
dirnensions  being  level  of  represo-.tation  (e.c,.,  parametric,  segmental,  syllabic,  lexical, 
syntactic),  tune,  and  alternatives,  with  contextual  ^nd  structural  support  connections 
explicitly  specified,  3)  a modular  structure  foi  incorporating  new  knowledge  into  the 
syslern  at  any  level,  and  4)  a system  strucUire  suitable  for  execution  on  a parallel 
processing  system 
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Ponormance 

The  present  system  has  been  tested  u in^  nnont  iOO  utterances  of  the  training 
data  tor  tite  1011 -word  vocabulary  task.  a grammar  with  simple  syntax  (AIX05. 
the  same  One  used  by  Harpy),  the  sentence  error  rate  is  about  167.  (semantic  error 
16/').  For  a grammar  with  more  complex  '■yntax  (AIX15,  see  appendic  III-C),  the 
centerice  error  rate  is  about  427,  (semantic  er-oi  26.')  The  system  runs  about  2 to  20 
times  slower  then  Harpy 

Discussion 

Hearsay-Il  represents  an  important  anc  continuing  development  m the  pursuit  ot 
large-vocabulary  speech  understanding  syste'-'s  The  system  is  designed  to  respond  in 
a semantically  correct  way  even  when  Iht  information  is  fuzzy  and  only  partial 
recognition  is  achieved.  Independent  knOA/le-ige  sources  are  easily  written  and  added 
to  Hearsay-!!;  knowledge  sources  may  ah  o ije  removed  in  order  to  test  their 
effectiveness.  The  Hearsay-!!  system  achitecture  offers  great  potential  for 
exploiting  parallelism  to  decrease  recogmtio  i times  and  is  capable  of  application  to 
other  knowledge-intensive  A!  problems  dealing  with  errorful  domains.  Many  more 
years  of  intensive  research  would  be  necessiry  in  order  to  evaluate  the  full  potential 
of  this  system. 


The  Locust  System  (Bisiani,  Greer,  Lowe  ire,  and  Reddy) 

Present  knowledge  representation  anti  searcii  used  in  Harpy  tend  to  require 
much  memory  and  are  not  easily  extendable  In  '/ery  large  languages  (v'Ocabulanes  of 
Over  10,000  words  and  more  complex  syrTav).  But  wc  do  not  view  this  as  an 
insurmountable  limilalion.  Modified  knowled(.c  representation  designed  for  use  wdh 
sc'conrlary  memories  and  specialized  pagirg  should  overcome  this  difficulty.  In 
addition,  it  appears  larger-vocabulary  speech  understanding  systems  can  be 
implemented  on  mini-computers  without  r ignif icant  degradation  in  performance. 
Locust  IS  designed  to  demonstrate  the  feasibihly  of  these  ideas. 

Model 

The  model  is  essentially  the  same  a ■ the  Hnrpy  systerr,  except,  given  the 
limitations  ot  storage  capacity  of  mam  memoi  y,  the  knowledge  representation  has  to 
be  reorganizud  significantly  . The  network  is  assumed  lo  be  larger  than  mam  memory, 
stored  on  secondary  men^ory,  and  retrieved  u ng  a specialized  paging  mechanism.  The 
choice  of  the  file  structure  representation  a icl  clustering  of  the  states  info  prages  of 
uniform  size  are  the  mam  technical  problems  associated  with  the  development  of  this 
system. 

Discussion 

' paging  system  for  the  ICl  1 word  ncabuia'-y  is  currently  operational  on  a 
PDP-1  l.MOE  and  has  speed  and  accuracy  perfnrmance  comparable  lo  Harpy  on  a PDP- 
10  (KAIO).  Simulation  of  various  oagmg  mod'  h is  currently  in  progress.  As  memories 
with  decreased  access  times  become  availaL  r,  this  class  of  systems  is  expected  to 
perform  as  accurately  and  nearly  as  fast  as  s ■ turns  requiring  no  secondary  memory. 


Parallel  Systems  (Feiler,  Fennell,  Lesser,  McCracken,  and  Oleinick) 

Response  ti.'C  for  the  present  systerr  is  usually  gresler  than  real-time,  with 
indications  that  larger  vocabularies  and  more  complex  s>  ntax  wTI  require  more  time  for 
search.  One  method  of  achieving  greater  spe -ci  is  to  use  para  iel  processing.  Several 
systems  designed  ana  developed  at  CMU  e |iioi:  rnulti-processor  hardware  such  as 
C.mmp  and  Cm*. 
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Models 

Several  systems  are  currer.tiy  under  development  as  part  of  multi-processor 
research  p»-0]ec‘s  which  attempt  to  explore  potential  parallelism  of  Hearsay  and  Harpy- 
like systems.  Fennell  and  Lesser  (1977)  stuemd  the  expected  performance  of  parallel 
Hearsay  systems  and  issues  of  algorithm  dec(imposition.  McCracken  (1977)  is  studying 
a production  system  implementation  of  the  H dr^a/  model.  OlcinicK  (1977)  and  Feiler 
(1977)  are  studying  parallel  decompositions  of  the  Harpy  algorithm.  Several  ot  these 
studies  are  not  yet  complete,  but  preliminary  performance  results  are  very 
encouraging.  Qleinick  has  demonstrated  a veision  of  Harpy  that  runs  faster  than  real- 
time on  C.mmp  for  several  tasks. 

Discussion 

The  main  contribution  of  these  system  studies  (when  completed)  will  be  to  show 
the  degree  of  parallelism  which  can  reasonably  be  expected  in  complex  speech 
understanding  tasks.  Attempts  to  produi  e reliable  and  cost-effective  speech 
understanding  systems  would  require  extensne  studies  in  this  direction. 


DISCUS;>10N 


In  the  previous  section  we  have  briefly  outlined  the  structure  and  contributions 
of  various  speech  systems  developed  at  CMli  in  retrospect,  it  is  clear  that  the  slow 
rate  of  progress  in  this  field  is  directly  attrib  itable  to  the  large  combinatorial  space  of 
design  decisions  involved.  Thus,  one  might  reasonably  ask  whether  the  human 
research  strategy  in  solving  this  and  other  • imilar  problems  can  benefit  from  search 
reduction  heuristics  that  are  commonly  used  m A!  programs.  Indeed,  as  we  look 
around,  it  is  not  uncommon  to  find  research  paradigms  analogous  to  depth-first 
exploration,  breadth-first  with  shallow  cut-oif,  oacktracking,  "jumping-to-conclusions", 
thrashing,  and  SO  on. 

Our  own  research  has  been  dominated  ny  two  such  paradigms.  First  is  a variant 
of  best-first  search:  find  the  weakest  link  (and  thus  the  potential  for  most 
improvement)  in  the  system  and  attempt  to  improve  it.  Second  is  a variant  of  the 
beam  searcin:  when  several  alternative  approaches  look  promising,  we  use  limited 
parallel  search  with  feed-forward.  The  systems  shown  m Figure  3 are  examples  of 
this  type  of  system  iteration  and  multi -system approach. 

Many  system  desig'’  decisions  require  i n operational  total  systems  framework  to 
conduct  experiments.  However,  it  is  not  r>'ct'ssary  to  have  a single  system  that 
permits  all  possible  variations  of  system  des  :',ns.  Given  enough  working  components, 
with  well-designed  interlaces,  one  can  construct  new  system  variants  without 
excessive  effort. 

The  success  of  the  speech  underst.-nd'ng  research  effort  is  all  the  more 
interesting  because  it  is  one  of  the  few  e •amplcs  in  A1  research  of  a five  year 
prediction  that  was  m fact  realized  on  lime  .ind  witnm  budget.  It  is  also  one  of  the 
few  examples  in  A!  where  adding  additional  k 'owlcdgc  can  be  uhown  to  lead  to  system 
speed-up  as  well  as  improved  accuracy. 

We  note  in  conclusion  that  speech  und  - standing  research,  in  spite  of  the  many 
superficial  differences,  raises  many  of  the  sa  'e  issues  that  are  central  to  other  areas 
of  AI  Faced  with  the  problem  of  reasoning  n the  presence  Of  error  and  uncertainty, 
we  generate  and  search  alternatives  which  iiave  associated  with  them  a likelihood 
value  representing  the  degree  of  uncertainty  Faced  with  the  problem  of  finding  the 
most  plausible  symbolic  description  of  the  utl-rance  ;n  a large  combinatorial  space,  we 
use  I c- c h r'liqijG $ suTiiJo*’'  «C  *hcc.c  used  Icoot'COrt  ^r^ph  sssrehing  prnQiom 
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solving.  Givfn  the  problems  of  acquisition  and  representation  of  knowledge,  and 
control  of  search,  techniques  used  in  spee' !'i  are  similar  to  most  other  knowledge 
intensive  systems.  The  main  difference  is  that  given  human  performance  the  criter'a 
for  success,  in  terms  of  accuracy  and  response  time,  far  exceed  the  performance 
requirements  of  other  AI  tasks  except  perhap'  vision. 
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II.  KNOWLEDGE  SOURCES  AND  TECHNIQUES 


The  Zapdash  Parameters,  Feature  Extraction,  Segmentation,  and  Labeling  for 
Speech  Understanding  Systems  (Goldberg,  Reddy,  and  Gill) 


Introduction 

In  spite  of  early  success  with  very  simple  parametric  representations  of  speech 
(see  Reddy  1966  and  Erman  1974),  recent  emphasis  has  been  on  highly  accurate  but 
computationally  expensive  parameter  extraction  techniques  such  as  LPC  spectral 
analysis,  formant  tracking,  etc.  We  feel  that  simpler,  more  efficient  methods  must  first 
be  applied  to  reduce  the  amount  of  input  data  before  more  expensive  analysis  is 
performed.  The  uniform  application  of  LPC  analysis  to  all  the  input  produces  accurate 
but  very  redundant  results,  ?nd  at  high  cost  (see  Goldberg  1975) 

Our  approach  involves  two  levels  of  parameter  extraction  and  analysis.  The  first 
level  produces  an  accurate  segmentation  with  strong  clues  as  to  manner  of  articulation 
and  phonetic  identity  of  the  segments.  For  this  purpose,  we  have  developed  the 
ZAPDASH  parameters,  described  below.  They  prcvide  a highly  efficient  basis  for  an 
accurate,  robust  segmenter  and  broad  classifier.  After  the  phonetic  elements  are 
isolated,  a uniform  LPC  labeling  stage  is  applied  only  where  it  is  needed  to  further 
refine  the  segment  identification.  Preliminary  evaluations  show  significant 
computational  savings  Is  possible  with  no  sacrifice  of  segmentation  or  labeling 
accuracy. 


The  ZAPDASH  Parametric  Representation 

As  digital  processing  ot  speech  becomes  commonplace,  it  becomes  desirable  to 
have  a parametric  representation  of  speech  which  is  simple,  fast,  accurate,  and  directly 
obtainable  from  the  PCM  representation  of  speech.  The  ZAPDASH  representation  of 
speech  (Zerocrossings  And  Peaks  of  Differenced  And  SmoolH  waveforms)  is  of  this 
nature.  An  important  means  Of  reducing  computational  cost  in  much  of  the  low  level 
processing  of  speech  is  to  reduce  the  quantity  of  data  in  the  input  representation  to 
the  minimum  necessary  for  accurate  analysis  of  the  phonetic  content  of  the  speech 
signal.  Our  oast  experience  shows  that  very  simple  measures  of  activity  in  the  low 
and  the  high  frequency  bands  (approximately:  <lkH2.  and  >lkHz.)  would  suffice  for  all 
but  the  fine  labeling  stage.  Peak-to-peak  amplitudes  and  zero-crossing  counts  provide 
simple  measures  of  the  amount  of  activity  within  each  particular  band.  In  ZAPDASH, 
the  PCM  data  is  used  to  generate  a differenced  waveform  and  a down-sampled, 
smoothed  waveform  (for  lOKHz  sampling  rate,  the  smoothing  FIR  filter  coefficients 
were  -1012444210  -1,  used  every  4th  point).  Peak-to-peak  distances  and 
number  of  zero-cf ossings  are  calculated  each  10  ms,  resulting  m 400  8-bit  parameters 
per  second  of  speecti.  ZAPDASH  can  be  calculated  in  15  to  20  computer  instructions 
per  sample  and,  therefore,  can  be  extracted  in  less  than  a 1/3  real  time  on 
minicomokiters  with  2 mic.o-sec.  instruction  time.  A simple  parametric  representation 
like  ZAPDASH  appears  to  provide  sufficient  information  for  accurate  phone 
segmentation,  thus  sharply  reducing  the  amount  Of  more  detailed  spectral  analysis 
required  by  n-iany  other  methods.  The  resultirig  four  parametric  measurements 
(Smoothed  Peak-to-peak,  Smoothed  Zero-crossing,  Differenced  Peak-to-peak,  and 
Differenced  Zero-crossing)  are  sufficient  to  detect,  with  reasonable  accuracy,  a set  of 
10  features,  described  below,  which  are  quite  useful  for  both  segmentation  and  initial 
brOad  labeling.  The  ZAPDASH  parameters  are  used  by  the  first  stage  segmenter  to 
make  decisions  on  manner  of  articulation.  The  resulting  segmentation  and  broad 
c ! 3 s 3if »c st lOrt  is  sccijrsts  yst  inci^pcyTsivc  Further  refinsmsnt  ot  the  ss^rrtsr^t  l0be!s 
using  spectral  analysis  is  then  much  more  economical. 


in 


SegmvnUtion  and  Broad  Clastification 

The  first  stage  of  the  prograrr.  contains  an  hierarchical,  feature-extraction  based 
segmer'iter  and  classifier.  A number  of  features  relating  to  manrser  of  articulation  are 
extracted  Silence,  voicing,  frication,  front-back  placement,  high-low  placement, 
consonant -like,  flap-like,  aspiration-like,  nasal,  and  sibilant  decisions  are  made  using 
the  ZAPDASH  parameters.  In  the  processing  of  an  utterance,  a sot  of  segments  is 
chosen,  with  broad  classification,  for  the  entire  utterance.  These  identify  regions  of 
the  signal  such  as  SIL-silence,  SON-sonorant,  UFR-unvoiced  fricative,  VBK-back  vowel, 
etc.  Further  sub-segmentation  and/or  reclassification  is  conditional  upon  segment 
class  type,  context,  and  feature  values.  There  are  59  classes  currently  used  internally, 
although  many  overlap  one  another  in  the  acoustic  space. 

Modified  LPC  Labeling 

At  the  second  stage,  where  no  further  refinement  is  possible  using  the  ZAPDASH 
information,  a fine  labeler  is  applied  at  the  mid-points  of  ail  segments.  The  original 
PCM  signal  is  compa'ed  against  stored  templates  by  a modified  LPC  distance  metric. 
ItaKura's  minimum  prediction  residual  metric  (Itakura  1975)  is  used  to  compare  the 
segment  mid-point  to  a set  of  speaker-specific  trained  templates.  The  segment  class  is 
used  to  p'ovide  a sub-set  of  the  approximately  100  templates,  or  a set  of  o priori 
weights  to  be  added  to  the  metric  values  for  all  templates.  In  this  way,  the  mannar- 
of-articulation  and  the  contextual  information  provided  by  the  earlier  feature 
extraction  improve  the  labeling. 

Results 

The  highly  efficient  segmentation  procedures  in  the  first  level  segmenter  and 
the  limitation  upon  the  need  for  LPC  analysis  provide  a factor  of  5 speedup  over  the 
uniform  procedures  used  by  HARPY  and  Hearsay-II,  Preliminary  tests  with  this 
program  indicate  that  results  for  HARPY  using  this  parameterization  will  be  just  as 
accurate  and  will  be  computed  faster  than  the  results  obtained  with  tne  more 
redundant  parameterization  it  now  uses.  Present  performance  of  ZAPDASH  can  be 
summarized  as  follows:  Segmentation  --  less  than  202  extra  segments,  less  than  27. 
missed  segments,  and  boundary  placement  within  an  average  of  10  ms.  of  the  manually 
defined  location.  Labeling  (broad  classes)  --  907.  correct,  (finer  labeling)  --  correct 
template  in  first  place  507.  of  the  time,  in  the  first  five  places  757.  of  the  lime.  A more 
detailed  evaluation  will  be  available  shortly. 
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A Syllable  Based  Word  Hypothesizer  for  Hearsay-11  (Smith) 

Problem  and  Motivation 

A central  problem  for  speech  understanding  systems  is  efficiently  and  accurately 
determining  what  words  are  implied  at  the  lexical  level  by  the  data  at  lower  levei^. 
One  solution  to  the  problem  is  to  map  each  word  hypothesized  by  syntactic  ar.r* 
semantic  information  to  the  lower  level  representation,  then  match  and  rate  the  word. 


But  as  speech  systems  permit  larger  vocabularies  and  languages  with  less  restricted 
syntax  and  sei  antics,  they  must  depend  more  on  bottom-up  methods  to  limit  the 
search  space  of  possible  word  sequences.  The  etfectiveness  of  a hypothesizer  can  be 
measured  by  the  percent  of  the  correct  words  and  the  number  of  competing  words  it 
hypothesiies.  One  method  of  bottom  up  word  hypothesization  is  to  go  directly  from 
the  phone  sequences  found  for  the  utterance  to  word  hypotheses  as  in  the  BBN  HWIM 
speech  system  (Klovstad,  1976).  The  solution  used  m Hearsay-I!  uses  an  intermediate 
level  of  syllables  between  the  words  and  phone  segments. 

Solution 

< The  word  hypothesizer  uses  equivalence  classes  of  syllables  (called  Syltypes)  to 
support  word  hypotheses  (Smith,  1976).  These  Syltypes  were  defined  so  that 
syllables  which  were  likely  to  be  given  similar  segments  and  labels  by  the  speech 
system  would  have  the  same  Syltype.  No  atlempt  is  made  by  the  word  hypothesizer 
to  distinguish  between  words  which  have  the  same  sequence  of  Syltypes.  The  word 
verifier  later  makes  this  distinction  as  it  rates  the  words. 

The  Syltypes  we  now  use  are  defined  by  a sequence  of  states  corresponding  to 
phoneme  equivalence  classes.  A Markov  probability  model  relates  the  state  sequence 
of  a Syltype  to  the  segment  labels  hypothesised  by  the  segmenter  and  labeler.  A 
word  may  be  hypothesised  by  the  following  sequence  of  events:  For  each  syllable 
nucleus  in  the  utterance  (defined  by  a heuristic  using  segment  labels  and  an  amplitude 
function),  the  most  likely  Syltype  state  sequences  are  found  by  searching  the  segments 
from  the  nucieus  out  to  adjacent  nuclei,  or  perhaps  the  utterance  boundaries.  For 
each  Syltype  hypothesized  with  a “good”  rating  the  set  of  words  containing  syllables 
mapping  to  the  Syltype,  are  retrieved  using  an  inverted  lexicon.  A multi-syllabic  word 
in  the  set  is  rejected  if  it  matches  poorly  with  adjacent  Syltype  hypotheses.  The  word 
verifier  is  then  called  lo  rate  each  word.  Those  with  a poor  rating  are  rejected. 

Results 

Since  the  word  hypothesizer's  ratings  (or  words  are  usee  only  to  determine 
whether  to  reject  the  word  or  to  verifier  the  word,  it  is  used  as  a filter  for  the  word 
verifier.  The  performance  relevant  to  this  task  is  the  percentage  of  the  spoken  words 
correctly  hypothesized  and  the  fraction  of  the  vocabulary  hypothesized  per  spoken 
word.  The  ret  s from  twenty  test  sentences  indicate  that,  for  a 1011  word 
vocabulary,  67/  of  the  correct  words  are  hypothesized  when  80  words  are 
hypothesized  per  spoken  word  (87.  of  the  vocabulary).  Of  course  these  numbers  can 
be  varied  by  changing  thresholds.  If  the  speech  system  can  function  with  only  577  of 
the  correct  words  hypothesized  boftom-up,  then  only  51  words  need  to  be 
hypothesized  per  spoken  word  (57  ol  the  vocabulary).  Similarly,  higher  accuracy  can 
b obtained  with  a greater  number  of  competing  word  hypotheses. 
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Wizard.  A Word  Verifier  for  Hearsay-ll  (McKeown) 


Problem  and  Motivation 

A 1.^.  . — ^ m ^ r‘  VIV  ^ M 

f-\  fNC/  prVl.ffC’lfl  u*  tUCI  .aiOMVJ'i  >■*  M It.  V C' • < • • v«li  • W/I  • 

hypotheses  generated  by  various  Knowleoge  sources  in  the  system.  The  verifier  must 
assign  8 likelihood  score  which  is  commensurate  with  the  match  between  the 
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underlying  acoustic  data  and  the  phonetic  description  of  the  #ord.  The  goodness  of  a 
score  may  be  only  temporally  significant;  the  scores  sho  jid  rank  order  competitive 
words  in  any  tiirie  area  such  that  the  correct  word  is  high  ii  the  ordering.  In  eddition 
to  this  acc  ?p1ance  criteria,  it  is  also  necessary  (or  the  verif  er  to  reject  absolutely  a 
largo  percentage  of  the  hypothesi?ed  words,  without  rejecting  a significant  number  of 
correct  words,  in  order  to  constrain  the  combinatorics  at  higher  levels. 

Solution 

In  HEARSAY  II,  words  may  be  generated  bottom-up  by  the  word  hypothesizer 
(POMOW)  or  predicted  top-down  by  the  syntax  and  semantics  module  (SASS).  Each 
uses  a very  different  strategy  for  verification  since  bottom-up  oypothesis  have  a 
known  approximate  begin/end  time  while  top-down  hypotheses  use  c verified  word  to 
predict  words  to  the  left  or  right,  and  thus  only  one  time  ir  known. 

Tlie  word  verifier,  WIZARD,  uses  a general  Markov  model  for  speech  recognition 
(BAKER, 1975  -,  LOWERRE,I976).  The  acoustic  information  is  a segme.Ttation  of  the 
utterance  where  each  segment  is  represented  as  a vector  of  phoneme  probabilities. 
Each  word  in  the  lexicon  is  represented  by  a statically  defined  ne-.work  which 
embod’es  alternate  pronunciations  of  the  word  This  model  f'nds  the  o,otimal  path 
through  the  word  network  and  assigns  as  the  word  score  a normalized  sum  of  all  the 
log-probabilit'es  for  states  (phonemes)  on  that  path.  Metworks  do  not  'ake  into 
account  word  junctures  but  do  handle  internal  phoneme  junctures.  Thus  WIZARD 
attempts  to  verify  words  as  if  they  exist  in  isolation. 

Wizard  handles  bottom-up  words  in  the  following  manner:  The  predicted 
begm/end  times  are  mapped  into  their  respective  begin/end  segments:  bseg/eseg  All 
paths  which  begin  at  bseg- 1 /bseg/beg+ 1 and  end  at  eseg-l/eseg/eseg  + 1 are  explored 
in  parallel.  Each  of  the  nine  possible  optimal  mappings  is  examined  and  the  best  of 
these  IS  chosen  as  the  mapping  of  the  word  network  over  the  segmented  acoustic  data. 
This  possible  time  shifting  allows  the  verifier  to  recover  from  incorrect  times  due  tc 
differences  m representation  of  the  acoustic  data  between  Knowledge  sources.  As  a 
result,  the  verif'er  may  change  times  on  word  hypotheses  as  well  as  rate  them. 

Words  which  are  hypothesized  top-dowi.  pose  a different  problem  in  terms  of 
verification,  since  only  the  begin  or  end  time  is  known.  In  this  mode  it  is  necessary  for 
WIZARD  to  predict  the  missing  time  as  well  as  to  return  a rating.  A major  problem  is 
bounding  the  number  of  segments  considered  in  a prediction.  Currently  several 
heuristics  a'T  employed.  Since  all  states  on  the  optimal  path  must  be  mapped  to  at 
least  One  segment,  the  tower  bound  on  the  number  of  segments  is  the  minimal  number 
of  network  transitions  (mintran)  An  upper  bound  was  experimentally  determined  to  be 
4»mintran,  thus  on  the  average  no  mere  than  4 segments  are  mapped  into  any  one 
state.  This  number  is  a function  of  the  segmentation,  which  tends  to  Over-segment, 
and  the  network  descriptions,  which  allow  reduced  spellings.  The  POMOW  word 
hypothesizer  p,ene''afes  an  upper  bound  based  on  the  expected  number  of  vowel 
nuclei  in  the  word  and  their  position  relative  to  the  beginning  of  the  prediction.  The 
smaller  of  these  uppe'’  bounds  is  used.  WIZARD  iteratively  maps  each  of  the  segments 
from  the  given  begin  segment  to  the  upper  bound.  It  considers  those  mappings  which 
fell  betw'een  the  lower  and  upper  bounds  and  picks  the  best  after  appropriate 
normalization  The  time  of  the  best  end  segment  is  returned  along  with  the  rating. 

Results  and  Conclusions 

The  results  summarized  in  Table  I are  tor  five  data  sets,  containing  100 
utterances,  in  which  332  correct  words  were  hypothesized  bottom-up  by  POMOW.  In 
addit-on,  13053  incorrect  words  were  generated.  The  vocabulary  size  for  POMOW  and 
WIZARD  was  approximately  550  words.  WIZARD  rated  each  of  the  words  using 
begm/end  times  generated  bottom-up.  Each  verification  took,  on  the  average,  100ms  of 
G^U  ^'^6*  3 DEC  ^DP"10  (KA,K  Fo*  ?3Ch  rsiing  (15,10)  0^ 

correct  and  incorrect  words  that  were  accepted  Or  rejected  is  tabulated.  From  this 
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data  the  number  of  words  hypothesized  per  word  position  and  the  percent  of  the 
vocabulary  hypothesized  per  word  piostt'on  can  be  calculated.  These  numbers  give  a 
vocebutary  independef^t  measure  of  performance,  allowing  comparison^  between 
various  system  configurations.  An  average  ranic  order  of  the  correct  word  is  provided 
which  measures,  at  each  threshold,  the  number  of  words  in  each  word  position  that 
must  be  examined  in  order  to  include  the  correct  word.  The  range  of  rank  orders 
between  the  data  sets  (20  utterances/set)  is  also  indicated. 


TR8LE  I 


IS 

$ HYPEO  8T  ponou 

ACCEPTEO 

REJECTED 

S.6  AANX  ORDER 

CORRECT 

332 

326  08/) 

6 ( 2/) 

(3.6  - 7.1) 

INCORRECT 

138S3 

18426  (88/) 

2627  (28/1 

TOTPL 

1338S 

187S2  (88/) 

2633  (28/5 

Ji/UORO  POS 

♦ 8 <80 

32  ( 6/) 

8 ( 2/1 

18 

» HYPED  By  POnOU 

ACCEPTEO 

REJECTED 

4.5  RAHi:  ORDER 

CORRECT 

332 

312  04/) 

29  ( 6/) 

(3.4  - 5.6) 

INCORRECT 

138S3 

6462  (49/) 

6S91  (SI/) 

total 

1338S 

6724  (SI/) 

6611  (49/) 

i/UORD  POS 

46  <8/1 

28  ( 4/) 

28  ( 4/) 

Sample  results  of  verification  in  the  prediction  mode  are  presented  m Table  II. 
In  this  mode  it  is  important  that  the  best  rating  for  the  predicted  word  comes  from  a 
mapping  that  closely  approximates  the  actual  time  in  which  the  wnrd  appears.  If  this  is 
not  the  case  there  is  the  danger  that  a correct  word,  which  is  highly  rated,  will  be 
hypothesized  with  times  which  will  disrupt  the  recognition  of  word  sequences  by  top 
end  knowledge  sources.  Small  errors  in  the  determination  of  the  missing  time  can 
propagate  time  errors  which  may  cause  whole  words  to  be  m.issed.  Table  II 
summarizes  the  results  of  an  experiment  to  predict  begin/end  times  of  529  words 
where  both  limes  were  actually  known.  The  d. stance,  in  segments,  is  calculated  from 
the  known  word  bound  and  its  predicted  word  bound.  The  table  also  shows  the 
distribution  of  distances  for  the  best  mapping.  Given  that  the  average  segment 
duration  is  3.2cs,  a distance  of  2 wou'd  correspond  fo  a range  of  predicted  bounds 
6.5cs  about  the  actual  bound.  Each  prediction  takes,  on  the  average,  ISOms  of  CPU 
time. 


TABLE  II 


BEST  RANKED 

PREDICTED 

JORO 

BOUNDARY 

DIST 

FREQ 

/ 

cun  / 

0 

125 

24/ 

247 

1 

209 

40/ 

64/ 

2 

103 

19/ 

83/ 

3 

41 

8/ 

91/ 

4 

28 

4/ 

95/ 

s 

17 

3/ 

98/ 

6 

7 

17 

99/ 

7 

4 

1/ 

108/ 

8 

0/ 

9 

1 

0/ 

18 

8 

Areas  of  fu/ther  research  involve  dynamic  generation  of  multiple  word  networks 


using  static  networks  and  word  jurvcture  rules,  alternate  score  norinalizet'on  schemes, 
and  improvement  in  the  effectiveness  of  bounding  predictions  using  vowel  nuclei. 
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Word  Pair  Adjacency  .Acceptance  Procedure  in  Hearsay-II  (Robert  Crook) 

Introduction 

In  the  Hearcay-II  speech  understanding  sys'em,  several  Knowledge  sources 
attempt  to  construct  sequences  of  words  from  the  word  candidates  hypothesized  on 
the  blackboard  Pairs  of  words  which  are  approximately  time -contiguous  and 
syntactically  adjacent  (may  be  paired  m the  grammar)  are  considered  for  extending 
word  sequences.  To  avoid  the  combinatorial  explosion  which  occurs  in  a grammar  with 
a large  branching  factor,  a procedure  is  required  which  will  constrain  the  number  of 
word  pairs  to  those  which  have  a high  probability  of  being  the  correct  ones. 

Such  a procedure  must  be  computationally  inexpensive,  since  it  must  make 
decisions  on  hundreds  of  pairs  of  hypothesized  words.  It  must  rely  upon  Knowledge  of 
word  junctures  and  upon  the  information  contained  in  the  segmental  transcription  of 
the  spoken  utterance.  And  it  must  reject  as  many  incorrect  pairs  (word  pairs  not 
actually  spoken)  as  possible,  without  rejecting  any  of  the  correct  pairs. 

This  paper  describes  the  word  pair  adjacency  acceptance  procedure  (JUNCT) 
developed  for  Hearsay-II,  the  knowledge  it  uses,  ^r.u  the  . 'l‘''■t  results. 

Description 

Input  to  the  JUNCT  procedure  is  a pair  of  woro  hypotheses.  If  it  determines 
that  the  words  are  adjacent,  based  upon  the  times  associated  with  the  hvpotheses,  the 
juncture  rules  contained  m the  procedure,  and  the  blackboard  segmental  description  of 
the  spoken  utterance  the  pair  is  accepted  as  a valid  sequence;  otherwise  it  is  rejected. 

Word  junctures  which  JUNCT  must  use  to  make  its  decisions  fall  within  three 
distinct  cases: 

(1)  Time -cont iguous  hypotheses:  Words  which  are  time  contiguous  in  the  blackboard 
are  immediately  accepted  by  JUNCT  as  a pss",ible  sequence  No  further  tests  for 
adjacency  are  performed 

(2)  Overlapping  hypotheses:  When  two  words  overlap  in  time,  juncture  rules  are 
applied  in  the  context  of  the  blackboard  segmental  transcription  of  the  utterance  to 
dete''mine  if  such  a juncture  is  allowable  for  the  word  pair 

(3)  Separated  hypotheses;  When  the  words  are  separated  by  some  interval  of  time, 
rules  arc  applied,  as  m the  overlap  case,  to  deferm.ne  whether  the  pair  can  be 
accepted  as  a valid  sequence  in  the  utterance. 

The  juncture  rules  used  by  JUNCT  are  of  two  types;  (1)  allowable  overlaps  of 
word  end-phoneme  and  begin-phoneme,  and  (2)  tests  for  disallowed  segments  within 
the  word  juncture.  A b't  .matrix  of  allowsuie  overlaps  is  precompiled  into  the 
procedure,  and  is  indexed  by  the  end-phoneme  and  begin-phoneme  of  the  word  pair. 
Any  overlap  juncture  involving  phonemes  which  are  not  allowed  to  share  segments  is 
rejected  by  JUNCT  In  the  separation  case,  as  in  allowed  overlaps,  the  blackboard 
segmental  description  of  the  spoken  I'tterance  is  examined  in  the  context  of  the  end- 
phoneme  and  begin-phoneme  oi  the  word  pair  to  determine  if  any  disallowed  segments 
are  present  m the  juncture  gap.  It  such  segments  are  found,  the  word  pair  is  rejected. 
Only  when  a word  pair  passes  all  rule  tests  which  apply  in  the  segmental  context  of  its 
juncture  IS  it  accepted  ac  r.  valid  sequence. 


Curr«nl  Results 

Stand-atone  performance  evaluation  runs  were  made  over  60  irttersnces  ustn^ 
words  generated  (rom  files  produced  oy  the  Hearsay-Il  word  hypothesizer. 
Syntactic  ally  adjacent  pairs  o*  words  whose  ratings  were  40  and  above  (on  a scale 
from  C to  100)  and  whose  times  (left-word  end  time  and  nght-word  begin  time}  were 
within  a 200  millisecond  interval  were  considered.  All  of  the  words  used  for  testing 
the  procedure  were  hypothesized  "bottom-up"  in  Hearsay-II;  no  predictions  we^e  used 
in  the  evaluation  runs.  The  following  table  summarizes  the  performance  of  the  JJNCT 
procedure. 


CORRECT 
WORD  PAIRS 

IN'CORRECT 
WORD  PAIRS 

total 

ACCEPTED 

188  (9b7) 

2S91  (417) 

3079  (42-7) 

REJECTED 

5 (5-1) 

4224  (597.) 

4233  (587) 

TOTAL 

197 

7115 

7312 

!t  IS  expected  that,  as  Icwer-level  sources  of  knowledge  provide  more  accurate 
times  for  word  hypotheses,  the  rules  for  acceptance  of  valid  word  pairs  may  be 
tightened,  further  increasing  the  speed  and  jserformance  of  Hearsay-It 


Syntactic  Processing  in  Hearsay-!!  (Hayes-Roth,  Erman,  Fox,  and  Mostow) 

The  bac'c  tasks  facing  the  three  syntactic  knowledge  sources  in  Hearsay-II  are- 
to  parse  synfacticaily  acceptable  sequences  of  words;  to  predict  words  that  can  be 
(syntactically)  adjacent  to  the  ends  of  a word  sequence;  and  to  construct  larger 
sequences  when  predicted  woros  are  verified.  The  chief  obstacle  is  finding  all 
possible  syntactic  structu-es  that  can  produce  a given  sequence  of  words.  Of  the 
traditional  parsing  n.echanisrr.s,  only  boftom-up  Kay-type  parsers  have  addressed  the 
problem  of  building  □■•Tase-sIructL.re  frees  which  are  not  necessarily  anchored  at  the 
start  (or  end)  of  a sentence.  But  these  methods  are  still  inadequate  lor  parsing  in  the 
current  environment  because  of  Ihe'r  requirement  that  all  constituents  of  a phrase  be 
present  m order  for  a phrase  to  be  recognized.  In  Hearsay-II,  a general  method  for 
such  partial  parsm.p.  of  incomplete  phrase  structures  nas  been  developed  and  is  used 
to  parse  grammatical  word  sequences,  to  predict  extensions,  and  to  join  up  to  three 
sequences  of  words  together  ir.  a new  syntactic  structure. 

The  details  of  the  method  are  now  briefly  described.  To  minimize  redundant 
computing,  the  syntactic  ^context-free)  grammar  is  converted  to  an  equivalent  template 
normal  form  grammar  m which  all  sequential  productions  have  binary  derivations  (e.g., 
B C D IS  replaced  by  A -*  3 X and  X -•  C D).  Thus,  frequently  occurring 
gramm.aiicai  subsequences  are  replaced  by  a common  higher-order  non-termina! 
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thereby  mnimizing  recomputation  of  common  subexpressions  (Hayes-Rot.h  and  Mostow, 
1975) 

The  word-sequence  hypothesizer,  WOSEQ,  generates  the  initial  word  sequer>ces 
that  are  partial-parsed.  Given  a word  sequence  wl  ..  wn,  the  RECOGNIZE  parser 
Knowledge  source  works  m a conventional  bottom-up  manner,  with  the  exception  that 
any  words  or  phrases  (r>on-termtni.ls)  that  are  required  by  a grammar  rule  to  precede 
(follow)  a constituent  at  the  first  (last)  position  of  the  sequence  are  pseudo- 
recognized;  that  is,  d the  x^ord  sequence  wl  ...  wn  can  be  derived  from  the  productions 
S -♦  A T,  T -*  wl  V,  V -*  U X,  U wn,  A -►  wO,  and  X -•  w(n+i),  then  the  non-terminals 
A and  X will  be  pseudo-recognized  and  the  sequence  wl  ...  wn  will  be  parsed  as  an 
instance  of  S,  with  closest  lett-misstng  constituent  A and  closest  right-missing 
constituent  X.  Bottom-up  parsing  continues  until  all  Of  the  words  in  the  input 
sequence  are  subsumed  by  each  highest -order  phrase  or  until  no  further  rewrites  are 
possible.  The  highest -order  phrases  constructed  that  derive  the  entire  word  sequence 
are  referred  to  as  spanning  phrases.  Because  parsing  is  discontinued  on  spanning 
phrases,  the  partial-parse  technique  essentially  identifies  minimal  (lowest -order) 
parses  of  each  sequence.  Each  distinct  parse  of  a sequence  specifies  a spanning 
phrase  and  the  pseudo-recogni:red  closest  missing  constituents.  There  may,  of  course, 
be  several  distinct  parses  of  any  word  sequence.  If  no  parse  of  a sequence  is  found, 
it  is  rejected.  Whenever  a sequence  hypothesized  by  the  word-sequence 
hypothesizer  is  rejected,  that  knowledge  source  wakes  up,  decomposes  the  rejected 
sequence  into  maximal  subsequences,  end  then  hypothesizes  any  sufficiently  rated  new 
word  sequences. 

Given  a spanning  parse  of  a sequence  wl  ...  wn  with  closest  left  and  right- 
miiSing  constituents  A and  X,  the  words  that  can  t>e  adjacent  fo  <wl  or  wn>  are  all 
rightmost  derivatives  of  A or  leftmost  cferivatives  of  X.  If  a spanning  phrase  has  no 
closest  !eft-missing  (right-missmg)  constituent,  the  possible  adjacent  words  are  found 
by  "going  up-and-over":  the  rightmost  (leftmost)  derivatives  are  computed  for  each 
constituent  that  can  be  directly  adjacent  to  this  left-complete  (right-complete)  phrase 
in  some  higher-level  spanning  phrase.  Predictions  of  words  are  made  by  the  PREDICT 
knowledge  source  whenever  the  extension  of  a previously  parsed  word  sequence  is 
scheduled  and  executed.  Predictions  may  be  made  to  both  sides  or  to  only  one  side 
depending  on  the  relative  and  absol"fe  numbers  of  grammatically  possible  words  on 
the  two  sides.  In  any  case,  if  none  of  the  predicted  words  on  one  side  is  verified,  the 
word-sequence  hypothesis,  although  syntactically  valid,  is  deactivated.  No  further 
processing  of  that  sequence  can  occur  unless  it  is  retrieved  by  another  sequence 
extension  colliding  with  it  on  the  side  that  failed  the  extension  effort.  Such  a salutary 
collision  results  in  toe  reactivation  of  the  sequence. 

When  predicted  words  are  verified,  the  CONCAT  knowledge  source  may  extend 
the  parse  by  concatenating  the  verified  words  to  the  predicting  word  reference. 
Given  the  sequence  sw  I ..  wn>  and  verified  preceding  predicted  words  al,  a2,  ...,  ak 
and  verified  succeeding  predicted  words  bl,  b2,  ...,  bm,  an  attempt  is  made  to  partial- 
parse  all  sequences  <ai  wl  . wn  bj>  as  well  as  ail  sequences  J x2  ...  xp  ai  wl  ...  wn 
bj  yl  y2  ...  yq^  where  sxl  x2  ..  xp  ai>  (■^bj  yl  yl  - yq^)  is  a previously  parsed 
sequence  of  words  on  the  blackboaro  that  is  time-adjacent  to  and  precedes  (succeeds) 
<wl  . wn>.  All  successfully  parsed  sequences  generate  phrasal  hypotheses.  Thus,  in 
addition  to  simply  extending  sequences  a-word-at-a-time  in  each  direction,  finding  a 
predicted  word  as  the  terminus  of  an  ex'sting  adjacent  sequence  can  trigger  the 
concatenation  of  three  sequences  at  once. 

Conclusion 

Because  the  words  that  are  hypothesized  from  other  knowledge  sources  form 
arbitrary  sequences  that  usually  do  not  completely  Sjtis'y  constituent  structures  Of 
phrase  rewriting  rules,  3 general  mechanism  for  partiai-parsmg  15  needed.  1 ne  current 
implementation  generates  minimal  spanning  phrases  and  retains  at  most  one  closest 


missing  constituent  on  each  side  o(  each  phrase.  Partial-parsing  times  average  about 
50  msec  on  the  KLIO  for  a 1000  word  vocabulary  with  a 15  branching-factor 
grammar.  Extensions  of  sequences  are  quickly  compuied  by  running  down  the  fight  or 
left  sor>s  of  the  binary  sequence  nodes  of  the  closest  missing  constituents.  Three 
adjacent  sequences  are  syntactically  concatenated  by  partial-parsing  the  concatenated 
word  sequences.  The  current  implementation  provides  an  efficient  solution  to  essential 
problems  of  syntactic  processing.  In  additicn,  the  three  related  knowledge  sources 
decompose  this  processing  into  natural  components  with  a grain-siae  that  is  attractive 
for  focusijpg  and  control. 
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Focus  and  Control  in  Hearsay-II  (Hayes-Rolh  anJ  Lesser) 

The  Hearsay-II  speech  understanding  system  currently  comprises  13  knowledge 
sources  (KSs),  11  <■  / which  are  data-direded  Each  data-directed  KS  is  invoked 
whenever  new  or  modified  blackboard  data  configurations  matching  patterns  of 
interest  are  found.  Monitoring  for  potentially  relevant  data  changes  is  performed  in 
two  steps:  changes  m hypotheses  or  links  at  particular  levels  are  collected  in  change 
sets  specific  to  each  KS:  procedures  called  preconditions  then  closely  examine  each 
accumufaled  change  and  its  blackboard  context  to  determine  if  the  exact  pattern  of 
interest  is  present.  Or>ce  such  a pattern  is  detected,  the  relevant  KS  is  invoked 
(scheduled)  to  operate  upon  it.  The  baste  control  problem  is  to  execute  first  those 
preconditions  and  KSs  that  are  most  likely  to  lead  to  successful  recognition  of  the 
utterance  The  two  chief  subgoals  are:  (1)  to  find  the  best  interpretation  as  quickly 
as  possible  and  (2)  to  reduce  the  number  of  incorrect  hypotheses  that  are  generated 
and  tested.  In  fact,  if  loo  many  incorrect  hypotheses  are  examined,  working  stor-age 
capacity  of  the  system  may  be  exceeded,  thus  precluding  eventual  correct  recognition 
of  the  utterance. 

The  current  approach  to  Ihe  control  problem  follows  closely  the  design  of  the 
focus  of  attention  mechanism  described  ir  detail  m Hayes-Roth  and  Lesser  (1976).  The 
basic  concepts  of  that  paper  are  quickly  reviewed  here;  (1)  The  Competition  Principle: 
the  best  of  several  alternatives  should  be  performed  first;  (2)  The  Validity  Principle: 
more  processing  should  be  given  to  KSs  operating  on  -nore  valid  data;  (3)  The 
Sip.nific ance  Principle:  mo'C  p'oeessing  snouW  be  given  Ic  KSs  whose  expected  results 
are  more  significant;  (4)  The  Efficiency  Principle:  mere  processing  should  be  given  to 
KSs  that  perform  most  reliably  and  inexpensively,  (5)  The  Goal  Satisfaction  Principle: 
more  processing  should  be  given  to  KSs  whose  responses  are  most  likely  to  satisfy 
processing  goals. 

The  degree  to  which,  a precondition  or  KS  sat  sfies  these  principles  is  reflected 
oy  its  desir ability,  an  increasing  function  of  its  validity,  duration,  level  of  analysis, 
importance,  concordance  with  control  thresholds  (goals),  (relative  and  absolute) 
expected  superiority  over  the  best  competing  alternative  in  the  same  lime  area,  and 
the  time  elapsed  since  an  improved  degree  of  recognition  was  achieved  (stagnation)  in 
that  time  area  While  the  desirab.hty  of  a KS  instantiation  awaiting  execution  is 
determined  directly  from  only  one  da*a  pattern  and  the  declarafi-'e  control  Knowledge 
about  fhe  direction  (on  the  blackboard)  and  relative  effectiveness  of  its  actions,  the 
desirability  of  a precondition  is  taken  to  be  the  maximum  of  such  values  over  all 
hypotheses  in  its  cnange  set. 

Using  this  general  scheme,  we  have  implemented  one  particular  control  strategy 
by  setting  particular  processing  goals  on  tne  blackboard.  Initially  fhe 


segmentr  r /Utjclier  i;-.  executed  and  i-j  'orted  i un  tc.  comptetiCn.  this  insures  that 
bnl(om-up  Syllable  h/potbesization  wili  have  the  benefit  o(  complete  segmental 
contexts.  The  syllable  hypothesiaer  is  executed  m turn,  and  for  a similar  reason  is 
also  forced  to  fcm  to  comptetron  At  this  &c:nt  the  Syllable -to -word  KS  responds  to 
new  syllables  and  generates  a!!  potentially  plausible  words.  The  strategy  module  then 
establishes  Ihresftoids  governing  whiCh  of  these  words  is  hypothesizec!.  It  attempts  to 
have  several  highly  rated  words  hypothesized  m each  area  of  the  utterance.  After 
this  processing  is  completed,  the  word-sequence  hypothesirer  examines  all  words  in 
parallel  and  identifies  promising  connected  sequences  of  time-adjacent  syntactically 
possible  pairs  of  words  (seeds).  The  best  of  these  m each  time  are  then  hypothesized. 
Krorr  this  point  on,  a complex  sequence  of  d:*ta-directed  preconditior»s  and  KSs  is 
invoked,  scheduled,  and  executed  to  control  syntact.c  parsing,  hypothesization  of 
plausible  words  to  extend  syntactic  sequences,  concatenation  of  verified  words  or 
phrases  with  adjacent  phrases,  and  the  generation  of  further  seeds  when  the  system 
is  stagnating.  Whenever  any  new  complete  parse  is  found,  a special  KS  is  invoked  to 
determine  which  remaining  hypotheses  and  KS  instantiations  are  insufficiently 
attractive  to  preserve.  These  are  either  rejected  or  deleted.  Processing  then 
continues  until  a quiescerce  occurs  reflecting  that  the  remaining  alternatives  are 
insuf f iciently  credible  to  continue.  If  a s ifficiently  p ausibte  sentence  has  been 
recognized,  the  stopping  condition  KS  deemes  to  terminate  the  analysis;  or  if  no 
complete  sentence  has  been  formed,  an  attemot  is  made  to  interpret  the  best  partial 
sequences  by  the  syntax  and  semantics  Knowledge  source. 

Conclusion 

Each  precondition  and  K3  's  regaroed  as  a [condition-faction]  schema,  with 
known  inputs  (blackboard  hypotheses  and  links),  a known  direction  of  action  (bottom- 
up,  top-down,  or  same-level  and  forwards,  backwards,  or  sarne-lime),  known  reliability 
and  efficiency,  and  therefore,  a Known  expected  result.  By  cornparmg  the  expected 
results  Of  all  scheduled  activities  to  the  ccu.'en!  state  of  recognition  and  desired  areas 
of  activity,  the  best  pending  instantiation  can  be  execued  first  As  a result  of  tuning 
the  various  weig.hting  factors,  we  seem  to  have  achieved  a desirable  balance  of 
breadth-  and  depth-first  search  (m  a giobai  sense)  with  effective  suppression  of  sub- 
optimal  (in  a local  sense)  activities.  Further,  by  separating  expensive  searches  into 
twe  or  more  successive  steps  (e.g.,  change  sets  and  preconditions  do  gross  filtering 
and  only  subsequent  KSs  do  fine,  expensive  processing;  or,  be'ore  expensive  syntactic 
searches  are  performed,  inexpensive  searches  are  made  for  plausible  sequences  Of 
syntactic  word  pairs),  it  appears  that  we  have  achieved  some  efficiency  in  the  overall 
organization  and  control  of  the  search  process 
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Policies  for  Rating  Hypotheses,  Halting,  and  Selecting  a Solution  in  Hearsay- 
II  (Hayes-Roth,  Lesser,  Mostow,  and  Erman) 

Purpiose  of  hypothesis  validity  ratings 

The  rating  policy  module  (RPOL)  m Hearsay-ll  provides  a uniform  basis  for 
comparing  the  plausibmty  of  different  hypotheses  The  hypotheses  may  be  competing 
alternative  interpretations  of  the  sair.e  portion  of  top  utterance  at  some  level  of  the 
blackboard,  in  wh  :h  case  the  nypoihesis  whu-.e  v^ii  J t y r at  .-r  is  higher  is  considered 


more  Itke)/  to  be  the  correct  interpretation.  Hov/ever,  the  hypotheses  may  describe 
different  portions  of  the  utterance,  or  provide  representations  at  different  levels  of 
the  blecKboard.  Having  a uniform  rating  policy  means  that  such  hypotheses  may 
nonetheless  be  meaningfully  compared  on  the  basis  of  their  validity  ratings.  This 
information  is  used  in  three  ways  by  Hearsay-II; 

(1)  to  locus  attention  m promising  directions  by  considering  Itigher -rated  (more 
likely  correct)  hypotheses  before  lower-rated  hypotheses.  This  is  implemented  by 
making  the  priority  of  a scheduled  action  an  increasing  function  of  the  validity  ratings 
of  the  hypotheses  which  are  being  acted  upon  (Hayes-Roth  end  Lesser,  1976).  Also, 
certain  types  of  actions  are  riot  even  scheduled  on  hypotheses  which  fail  minimum 
plausibility  tests  specified  by  knowledge  source  modules.  These  tests  use  'elidity 
ratings  as  a measure  of  plausibility. 

(2)  to  select  the  most  likely  correct  interpretation  of  the  utterance  if  there  is 
more  than  one  phrasal  hypothesis  spanning  the  utterance.  The  highest-rated  such 
hypothesis  is  then  the  chosen  interpretation. 

(3)  to  prune  the  sea'c'n  once  a solution  (i.e.,  an  utterance-spanning  phrasal 
hypothesis)  has  been  found.  This  is  done  by  restricting  further  processing  to  those 
actions  which  are  capable  of  leading  to  a better  (higher-rated)  solution. 

Computation  of  hypothesis  validity  ratings 

Hypotheses  in  Hearsay-ll  represent  'nterpretations  of  the  speech  signal  at 
various  levels  of  representation;  segmental  (lowest  level),  syllabic,  lexical,  word- 
sequential,  and  phrasal  (highest  level).  An  hypothesis  may  be  either  conjunctive, 
representing  a logical  product,  or  temporal  sequence,  of  lower  level  hypotheses  or 
disiunctive,  representing  a logical  summation  of  lower  level  alternative  hypotheses. 
The  degree  to  which  each  lower  level  hypothesis  supports  the  upper  hypothesis  is 
indicated  by  an  implication  between  -ICO  (maximally  disconfirming)  and  +100 
(maximally  confirming).  This  number  is  attached  to  a link  in  the  blackboard  from  ttie 
lower  to  the  upper  hypothesis. 

The  validity  ratinp,  VlCKH)  of  an  hypothesis  H is  a measure  of  the  extent  to 
which  that  hypothesis  is  supported,  ultimately,  from  the  acoustic  data.  The  lowest 
level  hypotheses  are  rated  by  the  bottom-end  processoi.  The  rating  of  a higher  level 
hypothesis  H is  computed  from  the  validities  of  the  hypotheses  which  support  H 
directly  from  below,  and  is  stored  on  the  blackboard  as  part  of  H.  The  validity  rating 
of  H need  only  be  recomputed  when  the  validdy  or  implication  of  its  support  changes, 
or  when  H receives  new  support  In  sum  cases,  RPOL  immediately  propagates 
resultant  validity  changes  up  through  the  blackboard.  Storing  the  ratings  on  the 
blackboard  avoids  the  expense  of  recomputing  them  recursively  whenever  they  are 
used 

The  validity  rating  VLD(H)  of  a disiunc live  hypothesis  H supported  by  n lower 
level  hypotheses  HI,  Hn  via  respective  link^  LI,  ...,  i n is  given  by 

Max  VLD(H.)*lMPLlCATl''<;g<Li)/100.  (l<i<n). 

Similarly,  the  validity  rating  of  a ccnpinr  bye  hypothesis  a!  the  word  level  Or 
below  IS  given  by 

(i  + (n-D/lC)  » (Sum  VLD<Hi)tlMPLlCATION(Li)/100),  (l<i<n). 

The  weighting  factor  <1  +rn-l)/10)  refects  the  increased  plausibihty  of  an 
hypothesis  which  has  mar./  .-cr./jncti/e  supporls. 

Above  the  word  level,  a soirewhat  ditfc  rent  function  is  used  to  rate  conjunctive 
hypotheses.  The  validity  VLD(H)  of  a phrasal  or  word  sequence  hypothesis  H is  given 
by  the  duration-weighted  average  validity  of  its  n underlying  words  Wi,  where 
duration  IS  measured  in  number  of  syllables.  J.e., 


VLCXM)  - (Sum  VLD{Wi)*length{W:))  / Sum  ler^gth(Wi),  (Isisn), 


where  length(Wi)  « length  (in  syllables)  of  the  word  hypothesis  Wi.  This  formula 
is  based  on  the  empirical  observation  that  the  longer  a word  IWi,  the  greater  the 
correlation  between  its  correctness  and  the  correctness  of  H. 

Halting  conditions  and  heuristic  pruning 

A phrasal  hypothesis  can  be  thought  of  as  a subpath  through  a flow  graph 
whose  arcs  are  word  hypotheses,  and  whose  souice  and  sink  are  respectively  the 
beginning  and  end  of  the  utterance.  A solution  (utterance-spanning  phrase)  then 
corresponds  to  a complete  path  through  the  graph.  The  validity  rating  of  a subpath 
(hypothesis)  is  given  by  the  average  arc  (word  hypothesis)  validity  along  the  subpath, 
weighted  by  arc  (word)  length  measured  in  syllables. 

There  is  a qualitative  difference  between  the  task  of  searching  for  a solution 
(complete  path)  and  the  task  of  deciding  when  to  stop  searching  and  accept  the 
current  best  solution.  The  former  task  can  efficiently  be  done  best-first,  t.e.,  by 
extending  the  most  promising  path  at  each  siep  in  the  search.  In  contrast,  the  latter 
task  inherently  involves  searching  all  possible  paths  in  order  to  guarantee  that  no  path 
is  better  than  the  best  one  found  so  far.  Once  a path  has  been  found,  the  goat  of 
processing  should  be  to  enable  such  a guarantee  to  be  made  as  quickly  as  possible.  In 
order  to  accelerate  the  attainment  of  this  goal,  two  heuristics  tor  pruning  the  search 
are  used. 

The  first  heuristic  consists  of  re lectine  every  word,  word  sequence,  and  phrase 
hypothesis  which,  due  to  its  low  rating,  cannot  be  extended  into  a better  solution  than 
the  best  already  found.  This  heuristic  can  be  thought  of  as  a form  of  alpha-beta 
prun  ng,  simplified  for  the  case  of  a One-player  game.  Rejecting  a subpath 
(hypothesis)  amounts  to  abandoning  certain  nodes  in  the  search  tree  which  correspond 
to  extensions  of  that  subpath.  In  operation,  an  hypothesis  is  rejected  if,  when  it  is 
extended  into  an  utterance-spanning  path  using  the  highest-rated  word  hypotheses 
currently  on  the  blackboard,  the  resuiting  (not  necessarily  syntactically  legal)  oath  is 
rated  lower  than  the  best  existing  solution.  Further  processing  on  rejected 
hypotheses  is  cancelled.  This  operalionalizalion  is  imperfect  in  that  it  ignores  the 
possibility  of  "missing  arcs,"  t.e.,  words  whicn  may  subsequently  be  predicted  by  the 
syntax  module  (added  as  arcs  in  the  graph)  and  be  rated  high  enough  to  invalidate 
previous  decisions  to  reject  earlier  hypotheses 

The  second  heuristic  is  based  on  the  observation  that,  if  a better  solution  than 
the  current  best  solution  exists,  it  must  be  possible  to  construct  it  by  extending  some 
existing  subpath  (hypothesis)  which  is  rated  higher  than  the  subpath  of  the  existing 
solution  spanning  the  same  time  interval.  (Once  again,  the  missing  arc  problem  is 
Ignored.)  All  hypotheses  (subpaths)  which  dc  not  have  this  property  are  deactivated, 
t.e.,  incapacitated  as  active  stimuli.  Any  scheduled  inferential  action  based  on  a 
stimulus  set  of  hypotheses  is  cancelled  if  all  the  hypotheses  in  the  set  are  deactivated. 
This  heuristic  can  be  though,  of  as  another  form  of  alpha-beta  pruning,  modified  to 
allow  sharing  of  common  subtrees  m the  search  tree.  Deactivating  a subpath 
(hypothesis)  amounts  to  deferring  expansion  of  certain  search  tree  nodes  which 
correspond  to  extensions  of  that  subpath. 

The  observed  effect  of  these  two  heuristics  is  to  cancel  a large  amount  of 
sctioduled  processing  once  a sviution  is  found,  and  to  focus  attention  on  those 
activities  which  are  capable  of  leading  to  a better  solution.  When  no  such  activities 
are  left  to  pursue,  RPOL  halls  processing,  selects  the  highest-rated  solution,  and 
passes  it  to  the  semantics  module  to  be  interpreted. 
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Solutions  and  partial  solutions 

RPOL  also  halts  processing  when  Hearsay-II  exceeds  predefined  limits  on  size  or 
execution  time.  In  this  case,  RPOL  chooses  the  highest-rated  utterance -spennirvg 
phrasal  hypothesis  as  its  solution.  If  no  such  hypothesis  has  been  generated,  RPOL 
tries  to  extract  a maximum  of  information  from  the  blackboard  by  selecting  the  best 
partial  parses  (phrasal  hypotheses)  and  pa-  smg  them  to  the  semantics  module  for 
further  interpretation  (Hayes-Roth,  Fox,  Gil!,  and  Mostow,  1976).  Here,  the  "best" 
phrase  hypothesis  H at  time  t is  considered  to  be  the  hypothesis  whose  time  interval 
includes  t and  which  has  the  highest  information  content,  defined  by  VLD(H)  » length(H). 
RPOL  finds  the  best  hypothesis  at  each  lime  t (measured  in  syllables  fr.om  the 
beginning  of  the  utterance),  and  passes  the  (typically  small)  set  of  such  hypotheses  to 
the  semantics  module.  Thus  even  when  Hearsay-11  fails  to  find  a complete  solution,  the 
best  partial  solution  (set  of  partial  interpretations)  is  found,  and  this  information  is 
used  in  determining  the  system’s  response  to  the  utterance  (Hayes-Roth,  Gill,  and 
Mostow,  1976). 

Conclusions 

The  task  of  rating  hypotheses  in  Hearsay-11  is  handled  by  the  system  policy 
module  RPOL.  The  role  of  knowledge  source  modules  in  this  task  is  limited  to  linking 
together  hypotheses  and  specifying  the  implications  with  which  lower  hypotheses 
support  upper  hypotheses.  Thus  the  effects  of  hypothesis  rating  changes  due  to  new 
information  are  automatically  propagated  throughout  the  blackboard  without  requiring 
the  help  of  the  knowledge  source  modules.  The  centralized  implementation  of  rating 
computation  and  propagation  has  made  it  easy  to  experiment  with  different  rating 
formulas.  It  has  also  simplified  the  task  of  developing  new  knowledge  source  modules. 

The  uniform  rating  scheme  employed  permits  the  meaningful  comparison  of  the 
plausibility  of  any  two  hypotheses.  Validity  ratings  are  used  by  Hearsay-II  to  focus 
processing,  to  prune  the  search,  and  to  select  the  best  solution  or  partial  solution.  In 
addition,  hypothesis  validity  ratings  are  used  by  the  knowledge  source  modules  for 
plausibility  tests  which  must  be  satisfied  in  order  for  various  inferencing  rules  to  be 
applied.  Thus  validity  ratings  help  to  guide  processing  in  a best-first  direction  until  a 
solution  is  found,  and  to  validate  it  quickly  thereafter  as  the  best  possible  solution. 
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Semantics  and  Pragmatics  in  Hearsay- H (Hayes-Roth,  Fox,  Gill,  and  Mostow) 

A speech  understanding  system  differs  from  a recognition  system  in  two 
principal  ways.  First,  an  understanding  system  verifies  that  the  sentences  it  hears  are 
meaningful  and  plausible  This  requires  use  of  semantic  knowledge.  Second,  the 
understanding  system  expects  particular  types  of  communication  to  occur  in  specific 
discourse  contexts  and  interprets  the  sentences  it  recognizes  accordingly.  Such 
expectation  and  contextual  interpretation  requires  pragmatic  knowledge.  The  purpose 
of  semantics  and  pragmatics  knowledge  sources  is  to  convert  this  knowledge  about 
meanings,  intentions,  and  communication  conventions  into  effective  action.  The  most 
significant  type  of  action  is  one  that  constrains  the  recognition  process,  a search  for  a 
plausible  parse  of  the  spoke'  utterance.  The  second  most  important  type  of  action  is 
to  hypoihesize  wliat  was  intended,  -when  wlra*  v.-as  said  cannot  fully  be  recognized. 
The  last  type  of  effective  action  needed  is  to  interpnet  (deduce  the  intention)  of  a 
successfully  parsed  utterance 
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The  complexity  of  artificial  spoken  languages  may  be  constrained  by  restricting 
either  the  way  ideas  are  expressed  (syntax)  or  the  number  of  ideas  that  can  be 
expressed  (semantics).  Our  approach,  in  the  news  retrieval  and  computer  science 
abstract  retrieval  tasks,  has  been  to  develop  one  comprehensive  semantic  grammar 
(average  branching  (actor  50)  used  for  interpretation  of  recognized  word  sequences 
and  to  vary  systematically  the  syntactic  con  trnmt  of  the  languages  used  for  speech 
recognition  per  se  (branching  factors  5,  15,  25).  Regardless  of  the  particular  syntax 
useo  for  recognition,  the  same  general  semantic  grammar  is  used  for  semantic  analysis. 
This  grammar  is  a template  grammar  like  those  developed  for  Parry  by  Colby,  with 
distinct  templates  for  each  unique  type  of  semantic  form  (Colby,  1974;  Hayes-Roth  and 
Mostow,  1975).  Semantic  interpretation  is  accomplished  by  extracting  from  the  (parse) 
tree  of  instantiated  templates  the  particular  words  or  expressions  filling  the  various 
functional  "slots." 

Parfi-lly  recognized  sentences  are  also  easily  interpreted  in  this  framework. 
When  the  attempt  to  recognize  a complete  sentence  has  failed,  the  best  (longest  and 
most  highly  rated)  syntactic  word  sequences  in  each  time  area  of  the  utterance  are 
passed  to  semantic  analysis.  All  templates  fully  or  partially  satisfied  by  word 
sequences  are  Instantiated.  The  most  fully  matched  semantic  pattern  is  then  chosen  as 
the  Interpretation  of  fhe  utterance.  Thus,  the  recognized  sequence  "Newell  or  Simon" 
would  be  interpreted  effectively  as  if  “List  ail  abstracts  by  Newell  or  Simon  from  any 
]Ournai  from  any  date"  had  been  recognized. 

The  capacity  to  provide  semantic  constraint  during  recognition  is  determined 
primarily  by  the  reliability  Of  predictions  regarding  what  the  speaker  is  likely  to  say. 
We  have  implemented  a discourse  knowledge  source  including  a conversation  model 
that  prompts  the  speaker  with  questions,  provides  information  about  using  the  system 
and  the  organization  of  the  data  base,  and  predicts  the  (semantic  and  syntactic)  type 
of  utterance  next  expected.  Earlier  versions  of  the  syntax  and  semantics  knowledge 
source  biased  recognition  actions  in  favor  of  predicted  communication  forms.  However, 
both  because  any  valid  sentence  is  permitted  at  any  time  and  because  the  system  is 
usually  employed  for  isolated  sentence  understanding,  no  direct  semantic  bias  is 
currently  used  The  basic  scheme  for  such  bias  is,  however,  conceptually  simple: 
given  an  expected  type  of  utterance  (a  higt’est -level  semantic  template),  recursively 
compute  the  expected  lower-order  subtemplates  and,  ultimately,  the  words  and 
pnrases  that  would  instantiate  the  expected  meaning  templates.  During  recognition, 
priority  is  given  to  actions  based  on  expected  forms,  at  the  expense  of  delayed 
processing  of  unexpected  word  sequences. 

Conclusions 

We  have  identified  three  types  of  actions  to  be  performed  by  semantics  and 
pragmatics  knowledge  sources:  (1)  bias  recognition  m favor  of  expected  forms:  (2) 
interpret  semantically  plausible,  partial  sequences;  and  (3)  correctly  interpret  the 
intention  of  the  speaker  when  a sentence  is  fully  recognized.  These  actions  are 
effected  in  Hearsay-II  by  combining  semantic  template  grammars  with  a conversational 
model  that  anticipates  the  speaker’s  general  intention  and  can  enumerate  its  manner  of 
expression.  The  realization  of  such  actions,  at  least  in  restricted  domains  Of  discourse, 
can  now  be  considered  a well-understood  technology. 
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Discourse  Analysis  and  Task  Performance  in  Hearsay-!!  (Hayes-Roth,  Gill, 
and  Mostow) 

The  discourse  analysis  module  (DISCO)  in  Hearsay-II  uses  knowledge  about  the 
state  of  the  conversation  to  interpret  the  speaker’s  intention  and  to  direct  the 
appropriate  actions  within  the  task  program.  Usually,  the  intention  of  the  speaker  is  to 
establish  a general  area  of  interest,  to  retrieve  articles  by  keyword  expression,  to 
further  qualify  a keyword  expression,  to  print  selected  articles,  or  to  request  certain 
information  about  the  retrieved  articles,  such  as  title,  date,  author,  author's  affiliation, 
or  publisher.  The  speaker  can  also  ask  for  help  or  complain  about  the  system’s 
response. 

The  state  or  discourse  is  represented  by  Ihe  contents  of  several  seir, antic 
registers,  one  of  which  points  to  a node  m a finite  state  automaton  discourse  model. 
(See  Figure  1.)  Each  node  in  the  model  corresponds  to  a general  sentence  pattern  or 
template  (Hayes-Roth,  Fox,  Gill,  and  Mostow,  1976).  (See  Figure  2.)  Other  registers 
hold  the  current  menu  (general  area  of  interest),  the  most  recent  keyword  expression, 
the  article  most  recently  referred  to,  the  most  recently  retrieved  articles,  and  the 
subset  of  retrieved  articles  which  satisfy  further  qualifications  specified  by  the 
speaker.  The  finite  state  model  is  used  to  interpret  yes-or-no  responses  and 
partially-recognized  utterances,  and  to  make  predictions  about  what  the  speaker  is 
likely  to  say  next.  All  possible  transitions  between  nodes  in  the  mo  'e'  are  permitted; 
the  arcs  in  the  model  indicate  the  transitions  which  are  considered  likely. 

Figure  3 shows  a sample  interaction  between  DISCO  and  a speaker.  Utterances 
enclosed  in  square  brackets  denote  recognized  spoken  utterances.  In  the  example 
shown,  the  first  utterance 


[ WE’RE  INTERESTED  IN  LEARNING  ] 

is  recognized  by  the  semantics  module  as  an  instance  of  the  ISELECTION  template,  and 
the  semantic  feature  SLEARNING  (indicated  area  of  interest,  or  menu)  is  extracted.  This 
semantic  interpretation  of  Ihe  utterance  is  passed  to  DISCO,  which  records  the 
indicated  area  of  interest,  LEARNING,  in  the  MENU  register,  and  sets  the  NODE  register 
to  point  at  the  8SELECTI0N  node  in  the  finite  state  model.  DISCO  then  predicts  that 
the  next  utterance  will  be  an  instance  of  the  SREQUEST  template  and  will  concern  the 
area  of  LEARNING.  These  predictions  can  be  used  to  bias  subsequent  processing  to 
favor  recognition  of  keywords  in  the  LEARNING  menu  and  function  words  characteristic 
of  a 8REQUEST  (Hayes-Roth,  Fox,  Gill,  and  Mostow,  1976).  Such  predictions  can  also  be 
used  to  respond  gracefully  in  the  case  of  a partially-recognized  utterance  (Hayes-Roth, 
Lesser,  Mostow,  and  Erman,  1976).  In  the  example,  if  the  speaker’s  second  utterance 

[ WERE  ANY  ARTICLES  ON  LEARNING  WRITTEN  IN  MAY  1374  ] 

were  not  fully  recognized,  DISCO  would  assume  that  the  speaker  had  REQUESTed  some 
articles  about  LEARNING  and  could  ask  him  to  repeat  the  request.  If  the  utterance 
fragment  "LEARNING  WRITTEN  IN  MAY  1974"  were  recognized  and  interpreted  by  the 
semantics  module,  DISCO  could  retrieve  articles  on  learning  dated  May,  1974. 
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KEYWOROEXP 


3Uf?RENT  ARTICLE! 


ARTICLES 


SUBSET 


Figure  1;  Semantic  registers  and  finite  state  discourse  model. 

labels  Y ar.d  N indicate  YES  and  ^iO  responses; 

0 indicates  empty  retrieval  set. 
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SStl.ECTlON  ' VJl'RE  1^‘TrRnSTrO  iN  l.EAHMII\;G  j 

Specifies  s tiifrna.  DISCO  respoi^ds  dy  pceitieg  Keywords  ai'^d  phrases  from  the  menu. 

Skl.OuLST  [ V.'LrRr  A,''JY  ARTICLES  O.'vi  LEAR.^iNG  VATITTEN  i!vi  MAY  1974  ] 

Spcicities  K set  of  articles.  DISCO  retrieves  the  articles  and  asks  for  further  directions. 

St^RUIvIlitLIST  [ Wi  llCl  l OF  TI-ILSE  ME.NTiON  nOEOTS  ] 

Further  specifics  «t  set  of  arlicies.  DISCO  removes  articles  from  the  currently  retrieved 
set  whicli  cion't  satisfy  the  new  restrictions. 

SGET.'ihji-O  [ Wi  Ij  WROTE  Ti  lCSE  ] 

Requests  iiifoi  .'nation  about  the  retrieved  articles.  DISCO  prints  the  requested  information. 

SLisrtnirM  f please  list  them  ] 

Requests  Output  of  a set  of  articles.  DISCO  prints  all  the  articles  in  the  currently  retrieved  set. 
ilLISTIlT  f PRIfijT  IT  ] 

Requests  output  of  a single  article.  DISCO  prints  the  article  most  recently  referred  to. 

sstop!!.15t:ng  [ STOP  LisTiriiD  please  j 

Requests  cessation  of  output.  DISCO  aborts  the  current  output  operation. 

SMAKE'FILE  [ MA<E  A FILE  OF  TitilSE  PIECES  j 

Requests  file  output.  DISCO  ci cates  a file  centa  r.:  '.3  the  retrieved  articies. 

SCOI'jTENTSfMENu  [ Wi  iAT  ARiE  ThE  KEYWORDS  •‘^OR  LEARiv;,'\iG  ] 

Requests  the  contents  of  a menu.  DISCO  lists  the  keyvrords  and  phrases  of  the  nienu. 

Siii.LP  [ V^iiA'!  CAN  I ASK  ] 

Requests  assistan.cc  or  u'.formal.on  about  tt'c  sysieiu.  DISCO  attempts  to  fulfill  the  request. 

SGiO^"  ■ V;i-IY  ARE  YOU  SO  SlCVV  ] 

Cempiaint.  DiSCO  responds  with  a pacifying  message. 

File  follbw  n;;  throe  nodes  represent  responses  to  yes-or-no  questions  asked  by  DISCO. 

iWAN  rsEil 

DISC'Cj  has  (irintcd  !iic  t ile  of  an  art.cie  and  asKea,  "Do  you  want  to  see  this  ai.icle?" 

SWA!')  I MOPE 

D!SCC>  iv.s  printer:  soir.r-  of  tnc  aroe'es  in  the  currently  retrieved  set  and  asked, 

"Do  you  want  to  see  anotiiC'  of  these  articles''" 

SMOREONI.'ENU 

Disco  nas  exnausted  tn.e  currently  .-elrievGa  set  ai'.d  asked,  "Do  you  want  to  ask  more  about 
<curri;iit  ri,cnu>’"  wncre  <currcnt  ii.cnu>  .s  the  .most  recent  area  of  interest. 


Text 


F rul  e 2.  1 eiv,p:,-,:ei  ccTe^ponoing  to  noces  i.i  tne  finite  state  model. 

Inst.iiKcs  o:  'cir.piates  a c snO'.vn  m square  oracKeis. 
describes  tre  -ntention  cf  tac;i  utterance  terripiate  and  DiSCO'c  response  to  it. 
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OK.  Vi/i-iA T SLiiiJI  Ci  tX)  YOU  VJAr>Ji'  TO  ASK  ABOUT? 


[ VJL'iit  INTLRCSITO  IN  LEARNING  ] 

i t-.t;  KiiYwoi;';:)  PiT..ASt:s  v^t hcit  i Krjov;  about  are; 

LEAlJi.iNG 

hypc  ; ;::  sis  format iON 

cell  AOSTMOLY  Tr.EORY 

(JkAIN  iT.i:ORY 

CAUOAL  REASONiNG 

CO'/MOU  SENSE  Ti-TIORY  FORMATION 

CjNOrPTUAi.  INFERENCE 

CYiiKRNFriCS 

grammatical  ineerence 

NEUIJAi.  Ni  TVJORVS 
PERCEPT ROMS 

PijLh  10'-;  CAPABlLlTitS 

ABSTRACTiON 

ACC'UlCi  1 iv.)N  Or  KNOVi'LEOGE 
OK,  ASK  ME  ABOUT  LEARNING 


[ Vu'i  Ri  ANY  ARTICLES  ON  LEAPNiNC  VARITTEN  IN  MAY  1974  ] 


TPi'RE  API;  > STORIES 
7i  !E  iPRST  ONE  IS: 

Til  IE: 


L-. . ERAu  1 tVii  lIvVSR 


KOCOi  S Wi  i;C!-i  CAN  i.EAPN,  PLAN  THEIR  ACTIONS,  A 


V Ti  BEHAVIOR  or  SEVERAL 
NO  COEXIST 


IXj  you  WAi'.'l  TO  SEE  iHiS  ARTICLE':' 


[ V>/I--::.  VACO  iE  il'IESE  J 


i'ESH'i'J  C'E  AI,  il’i I i.RAC r IVL  ENV): 
PCiUOTS  Vv'HlCi  I CAl'i  lEAPN,  Pi.AN 
NiCl  iE'LAS  V.  FiNLY..;  P 


T.'ENT  TO  Si'uOV  Ti-,::  BEhAVIOR  OF  SEVERAL 
. :R  mC  i IGNS,  ANi.^  uOEXii>  i 


pr',jCY  i.E AP^^jl^‘,C  il',  TVv ‘V.  Cl  vj:  r<\jOOTS 

lAUPENT  SiKwOSSY 


i L‘  3. 
SpOKcn 


SHir.pin  .r.Tei  rictiOn  oeivveon  OISCO  <ii'.cj  ?.;)C'rtker. 
jltcra'xcr-  Ate  crcloicc  111  squcrc-  or.icuc-ts. 
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PARALLEL  PRtXESSING  IN  SPEECH  UNDERSTANDING  SYSTEMS 


Parallelism  in  Artificial  Intelligence  Problem-solving  (Lesser  and  Fennell) 

Background 

Many  artificial  intelligerKe  problem-solving  tasks  require  large  amounts  of 
processing  power  because  of  the  size  of  the  search  space  that  needs  to  be  examined 
during  the  course  of  problem  solution.  This  :s  especially  true  for  tasks  that  involve 
the  interpretation  of  real  world  perceptual  data  which  is  generally  very  noisy  (i.e., 
speech  and  image  understanding  systems).  For  example,  a speech-understanding 
system  capable  of  reliably  understanding  connected  speech  involving  a large 
vocabulary  is  likely  to  require  from  10  to  100  million  instructions  per  second  of 
computing  power,  it  the  recognition  is  to  be  performed  in  real  time.  Recent  trends  in 
technology  suggest  that  raw  computing  power  of  this  magnitude  can  be  economically 
obtained  through  a closely-coupled  network  of  asynchronous  “simple"  processors.  The 
ma]0r  problem  with  using  a network  multiprocessor  Is  in  specifying  the  various 
problem-solving  algorithms  in  such  a way  as  to  exhibit  a structure  appropriate  for 
exploiting  the  available  parallelism. 

This  restructuring  Of  an  artifiicial  intelligence  task  for  parallel  processing  may 
not  be  as  difficult  as  might  be  expected.  The  basic  problem-solving  paradigm  that  is 
used  to  resolve  ambiguities  resulting  from  the  error  in  input  data  and  the  imprecise 
and  orrorful  nature  of  knowledge  sources  implicitly  involve  parallel  activity.  This 
parallel  activity  arises  because  many  weakly  supported  alternative  hypotheses  must  be 
"simultaneously"  evaluated  m order  to  locate  a consistent  hypothesis  which  is  a 
solution  to  the  problem.  These  problem-solving  techniques  are  implemented  through 
sophisticated  control  structures  that  (1)  permit  the  selective  searching  (usually 
heuristic)  of  a large  part  of  the  state-space  of  possibilities  and  (2)  allow  the  combining 
of  multiple,  diverse  sources  ot  knowledge  (e.g  , in  the  speech  domain,  acoustics,  syntax, 
semantics,  prosodies)  so  as  to  cooperate  In  resolving  ambiguity  [Reddy  76,  Woods  74, 
and  Lesser  75A}.  The  state-space  searching  m existing  systems  is  implemented 
through  backtracking  control  structures;  these  are  basically  sequential  implementations 
of  non-deterministic  control  structures.  Thus,  a 'arge  potential  fo'-  parallelism  arises 
from  implernentmg  these  non-deterministic  control  structures  in  a parallel  manner,  i.e., 
searching  different  parts  of  the  stale  space  m parallel.  In  addition,  these  diverse 
knowledge  sources  (KS’s)  can  be  made  independent,  there  exists  the  potential  for  a 
proportional  speed-up  m the  recognition  process  by  executing  them  in  parallel. 
Finally,  ttiere  is  the  possibility  of  decomposing  each  knowledge  source  into  separate 
paralle'  processes. 

Summary  of  Currant  Raseareh 

In  order  fo  test  the  ease  and  effectiveness  with  which  an  artificial  intelligence 
task  could  be  structured  for  and  executed  on  a multiprocessor,  an  organization  for  a 
knowledge-based  artificial  intelligence  problem-solving  system  was  developed  which 
takes  maximum  advantage  of  any  separability  of  the  processing  or  date  components 
available  wthm  that  organization.  Knowledge  sources  are  intended  to  be  largely 
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independent  and  capable  of  adynchronous  e>ec'jtien  m Ihe  fcm  of  Knowledge  source 
processes.  ''>/era!l  system  control  is  distributed  and  primarily  data-directed,  being 
based  on  events  occurring  in  a globally  sbared  data  base.  Such  a problem-solving 
organization  is  believed  to  be  particularly  amenable  to  implementation  in  the  hardware 
en.'-ronment  Of  a network  of  closely-coupled  asynchronous  processors  which  share  a 
common  memory.  The  Hearsay  II  speech-unde'standing  system  (HSU)  [Lesser  75, 
Fennell  77,  Erman  75],  which  has  been  developed  using  the  techniques  for  system 
organization  described  above,  has  provided  a context  for  evaluating  the 
multiprocessing  aspects  of  this  system  architecture. 

Based  on  multiprocess  simulations  and  implementation  of  these  systems  on  the 
C.mmp  multiprocessor,  the  following  results  were  Obtained  [Fenroell  75]: 

1.  There  does  exist  extensive  p3rallr>lism  in  the  speech  understanding 
task  (e.g.,  given  a small  configuration  of  Knowledge  sources,  between 
4-14  processors  could  be  effectively  ut  lized). 

2.  The  overheads  involved  in  supporting  the  multiprocessing  and 
synchronization  primitives  are  quite  high  (eg..  Over  1002). 

3.  The  locking  structures  had  to  ve  very  carefully  tailored  to  the 
particular  set  of  knowledge  sources;  otherwise,  the  effective 
parallelism  would  be  significantiy  degraded. 

In  trying  to  understand  the  implications  of  tiie  last  two  results,  some  tentative 
observations  were  made.  The  first  and  somewhat  su.  prising  observation  was  that  the 
basic  self-correcting  nature  of  tne  information  flow  m the  HSIl  system,  which  comes 
from  knowledge  source  cooperation  through  a hypothesi?e-and-lest  paradigm,  may 
Ob  viate  the  need  for  most  uses  of  explicit  synchronization  techniques  to  maintain  data 
integrity  To  elaborate  on  this  point,  one  knowledge  source  can  correct  the  mistake  of 
another  knowledge  source  whether  the  error  arises  from  a mistake  in  the  theory 
behind  the  Knowledge  source  or  from  incorrect  synchronization  (i.e.,  working  on 
partially  invalid  data).  Another  example  of  this  self -correcting  type  of  computation 
structure  is  the  relaxation  method  (iterative  refinem.ent)  used  to  solve  partial 
differential  equations  This  type  of  computational  structure,  when  put  on 
asynchronous  multiprocessors,  can  be  decomposed  so  as  to  avoid  a lot  of  explicit 
synchronization  at  the  expense  of  more  cycles  for  convergence.  This  type  of 
decomposition  is  accomplished  by  not  requiring  each  point  to  be  calculated  based  on 
the  most  up-to-date  values  of  its  neighboring  pents.  The  iterative  refinement  njiture 
of  computation  will  correct  (within  a certain  range)  for  this  lack  of  synchronization,  it 
IS  felt  the  feed-forward/feed-backward  dala-directed  problem-solving  paradigm  of 
HSII  haj  similar  properties.  The  other  observation  was  that  a drastic  decrease  in  the 
cost  of  certain  types  of  synchronization  primitives  could  be  accomplished  if  their 
implementation  is  tailored  to  their  (statistical)  usage. 
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The  HSll/C  mmp  System  (Lesser,  Buchalter,  McCracken,  Robertson,  and 
Suslick) 

The  HSH/C.mmp  system  has  been  developed  to  test  whether  an  asynchronc.-s 
multiprocess  architecture  such  as  C.mmp  (Id  PDP-11  processors  sharing  a common 
memory)  can  be  effectively  applied  to  speed  up  t.he  higher  level  processing  of  a 
speech  understanding  system.  Extensive  sin.ulation  studies  'vere  done  on  a PDP-10 
using  a multiprocess  version  of  '-tearsay-II  to  test  the  feasibility  of  the  idea  before 
embarking  on  the  actual  implementation  (Fennell  and  Lesser  1977). 

A prototype  version  of  this  system  written  :n  L»,  a system  building  language 
developed  by  Newell  et  al.  1970-71,  was  con*  tructed  and  running  in  February  of  1976. 
In  addition,  an  algebraic -language  interpreter,  SL*,  was  constructed  for  executing 
knowledge  sources  written  in  an  Algol  dialect.  However,  the  knowlec'j.e  source 
modules  were  very  primitive,  and  no  substantial  results  were  obtained  except  the 
measurement  of  the  overhead  of  certain  Hearsay-1!  primitives.  As  a result  of  tnese 
measurements,  a reimplementation  was  begun  m order  to  significantly  speed  up  the 
system  (especially  those  system  primitives  which  deal  with  syn: hromzaf lOn  operations), 
and  to  make  it  possible  to  run  large  knowledge  source  module^  in  the  small  address 
space  environment  that  the  POP- 1 1 provides.  This  rfimpieinenia^'OrT  is  now  almost 
complete,  with  preliminary  results  indicating  a speed-up  of  approximately  10  over  the 
original  version  In  addition,  a translator  has  been  deve’oped  which  takes  most  PDP-10 
statements  written  in  SAIL  and  translates  them  in!o  equivalent  SL»  statemenis.  Thus,  it 
should  be  possible  in  the  next  few  months  to  run.  vvilhout  major  code  conversion,  the 
knowledge  source  modu'es  of  the  POP-10  Hearsay-tl  system  on  *he  HSlI/C.mmp 
system. 
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A Parallel  Production  System  (or  Speech  Understanding  (McCracken) 

The  queslion  addressed  bv  f is  thesis  {McCracken  1977)  is  whether  or  not  a 
production  system  architecture^  can  remedy  sorrie  ot  the  chronic  problems  o( 
knowledge  representation  and  system  crgani  abon  in  large  knowledge-based  artificial 
intelligence  systems,  particularly  speech  understanding  systerr.s.  0(  particular  interest 
is  the  problem  ot  exploiting  parallel  machine  architectures  to  obtain  near  real-time 
response.  To  explore  this  question,  a production  system  version  of  the  Hearsay-ll 
speech  understanding  system,  called  HSP,  for  HearSay  Production  system,  is  being 
imple.Tiented  on  C.mmp,  the  CMU  multi-mini-processor.  A large  fraction  of  the  Hearsay- 
it  speech  knowledge  has  been  translated  into  productions  for  HSP,  specifically: 
POMOW  (word  recognizer),  POSSE-WOMOS  (word  verifier)  and  SASS  (syntax  arid 
semantics)^. 

Expected  results  come  under  two  mam  categories;  comparisons  between  the 
way  knowledge  is  encoded  in  HSP  versus  Hcarsay-II,  and  comparisons  in  the  use  of 
parallelism.  The  major  differences  between  the  HSP  and  Hearsay-ll  architectures  are: 
(1)  the  basic  knowledge  unit  in  HSP,  a production,  is  considerably  smaller  than  a 
Hearsay-II  Knowledge  Source  ; (2)  HSP  encodes  knowledge  m a more  formal  and 
simple,  but  less  expressive,  language  than  Hcartay-Il;  (3)  HSP  totally  segregates 
condition  from  action  (i.e.,  read  from  write),  white  Hea'^say-Il  allows  a mixture;  and  (A) 
there  is  virtually  no  use  o(  local  working  meniOry  m HSP  (only  a single  shared  working 
memory),  whereas  Hearsay-ll  knowledge  sources  make  use  ol  rather  laige  local  data 
contexts  in  addition  to  the  shared  Blackboard.  It  is  expected  that  these  architectural 
differences  will  yieid  an  improvement  for  HSP  m effective  parallelism,  in  clarity  of 
knowledge,  in  ease  ol  augmentation,  and  m other  problem  areas,  such  as  handling  of 
error,  directionality  control,  and  performance  analysis. 


1.  A production  system  encodes  all  long-te:m  knowledge  as  simple  condilion-aclion 

rules  which  operate  from  a shared  wording  memory.  For  entry  into  the  subject 
see:  R.  Davis  and  J.  King,  An  Overview  o(  Production  Systems,  Computer  Science 
Department,  Stanford  University,  Oct.  1975. 

2.  POSSE,  WOMOS,  and  the  version  of  SASS  used  are  from  an  earlier  version  of 

Hearsay-II  used  m the  Spring  of  1972. 
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III.  PERFORMANCr  MEASUR£ME^f^ 


In  this  section  we  present  the  detailed  perforwarKe  resuits  obtained  for  the 
Msrpy  af>d  Hearsay-I!  systems  in  September  of  1976.  Sir*ce  then  both  systems  have 
been  improved;  future  papers  will  provide  results  of  improved  performance.  The 
purpose  of  this  secliori  is  to  provide  a record  of  system  performaixe  as  measured  on 
September  8.  1976 

!n  addition  to  the  performance  of  the  systems  on  the  1011-word  tasks,  this 
section  also  contains  results  of  experiments  on  connected  digit  recognition,  effect  of 
telephone  on  accuracy,  effect  of  multiple  speakers  (using  speaker  irxieperxtont 
templates)  on  accuracy,  and  effects  of  branching  factor  and  vocabulary  size  on  the 
performance  of  the  Harpy  system. 


PerformafKB  of  the  Harpy  and  Hearsay-ll  Systems 

Figure  1 gives  the  performance  of  ti>e  Harpy  system  on  the  1011 -word  AI 
abstract  retrieval  task.  The  vocabulary  used  m this  task  and  the  phone  dictionary 
associated  with  the  vocabulary  is  given  in  Appendix  III-B, 

Given  the  vocabulary  and  protocols  taken  of  humans  interacting  with  a mock 
system,  Hayes-Roth  generated  a set  ol  typical  sentences  that  are  likely  to  be  useful  in 
the  abstract  retrieval  task  No  attempt  was  made  to  restrict  these  to  any  specific 
grammar.  However,  care  was  taken  to  see  that  each  word  in  the  vocabulary  occured 
-it  least  once  in  these  sentences  These  sentences  (a  total  of  ^96)  served  two 
nurpeses:  1)  as  a set  of  training  sentences  (spoken  by  Lee  Erman),  and  2)  for  the 
design  of  a family  of  languages  with  varying  branching  factors  that  accept  at  least  the 
;raining  sentences  and  possibly  many  more. 

Goodman  designed  n.any  such  languages  Two  extreme  examples  are  a language 
where  any  word  (of  the  1011)  could  follow  any  other  word,  permitting  many  nonsense 
sentences,  and  another  in  which  only  the  496  training  sentences  were  legal.  Of  the 
several  languages  errasen  for  the  experiment i hon,  three  specific  ones--AlX05,  AIX15, 
and  AlXE--are  given  in  Appendix  lil-C  (an  earlier  version  of  AIXF  was  developed  by 
Kayes-Roth). 

The  grammar  that  allov-ed  Harpy  to  reach  the  performance  goals  of  the  ARPA 
program  was  A1X05.  with  a static  branching  factor  of  9.53  and  an  average  dynamic 
fanout  of  33.4.  The  others  were  too  large  to  fit  within  the  memory  of  the  PDP-10 
system,  however,  it  was  possible  to  study  the  performance  ol  AIX15  and  AIXF  using 
variants  which  used  smaller  vocabularies,  created  by  eliminating  some  of  tho  proper 
nouns. 

The  training  sets  for  the  other  foui  speakers  (two  male  and  two  fjmale) 
consisted  of  a small  subset  of  the  original  training  sentences.  These  were  used  to 
generate  speaker-dependent  phone  template;  fer  each  of  the  speakers  (see  the  paper 
by  Lowerre  in  SectionE  on  speaker  adaptaticn). 

A completely  new  set  ol  100  test  seninnces  was  created  by  Hayes-Rofh  which 
were  not  part  of  toe  training  set.  These  are  given  m Appendix  III-A.  Erman  recorded 
all  the  100  test  sentences  and  the  other  four  speakers  recorded  a subset  of  twenty 
one  sentences  each.  These  sentences  were  used  only  for  testing  the  performance,  of 
the  system;  the  system  was  not  tuned  m any  way  m response  to  errors  in  this  set. 

The  Harpy  system  achieved  an  aggregate  9i>’  sentence  accuracy  and  95t( 
semantic  accuracy  Over  all  the  5 speakers  end  required  27.9  million  'nstruchons  per 
second  of  speech  processed  (Fig  l).  Hears, *y-H  (Fig  3)  was  tested  on  only  twenty 
two  sentences  tor  lack  o*  tirr.e  and  achieved  917  semantic  accuracy  and  required  about 
85  mipss.  Figures  2 and  4 give  the  performance  of  the  two  systems  on  test  sentences 
recorded  live  in  the  classroom  on  September  8 The  Harpy  system  recognized  four  of 


12 


the  five  sentences  recorded  by  two  male  and  one  female  speaker  correctly.  The 
Hearsay-II  system  recognized  three  Of  the  five.  These  senterKes  were  generated  by 
the  observers  who  were  givert  copies  of  the  grammar;  the  seritences  were  in  no  wey 
preselected.  The  classroom  environment  was  somewhat  more  noisy  than  the  terminal 
room  environment  normally  used  to  collect  training  data. 
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TASK 


Recognition  of  AI  i ntormat ion  retrieval  task 
Vocabulary  size:  1011 
Branching  factor:  3.53 
Average  fanout:  33. A 


DATA 


Number  of  speakers:  5 
3 male 
2 female 

Training  set  for  speaker  LE 
sentences 
40A9  uords 

24.7  minut'>s  ot  speech 

for  5:''-,akers  DS  KP  BH  GUI 
sentences 
1444  uords 

10.1  minutes  of  speech 


Training  set 
256 


Test  set  for  all  s.:>eakers 
184  sentences 
1138  uords 

6.5  minutes  of  st^eech 


PERFORnANCE  ON  THE  TEST  DATA 
37%  word  accuracy 
31%  sentence  accuracy 
95%  semantic  accuracy 
27.9  Hi pss 


Figure  1.  Harpy  results  for  the  A!  retrieval  task  test  data. 
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RES'iLTS  OF  IVE  SENTENCES 


HfiRPY  VERSION 


UVT 

T,?12IN  UOROSIN 

UOROSOUT 

JCOR 

TCOR 

TIME 

TinEOUT 
/TIRE  IN 

fSTpTEs 

SECnENTS 

1 

2.2 

6 

6 

6 

me.o 

71.3 

32.2 

261 

82 

2 

2.1 

6 

6 

6 

me. 8 

69.9 

33.8 

355 

70 

3 

3.8 

9 

11 

5 

55.6 

381.1 

88.3 

396 

138 

4 

2.1 

9 

9 

9 

iro.  B 

96.1 

45.6 

432 

81 

c 

1.5 

4 

4 

4 

lOO.B 

53.  B 

35.6 

352 

53 

2.3 

34 

36 

38 

.68.2 

118.3 

56.8 

S.O.. 

359 

18.8 

36.41 

Correct  utts*4/5  * 60.6/ 


RESULTS  OF  LIVE  SENTENCES  HRRPY  VERSION 

UTT  1 

UTT="RRE  RNY  PfiPERS  PBOUT  SEnOMIC  NETUORI3" 

RECs'RRE  RNY  PfiPERS  fiSOUT  SEnPNTIC  NETUORIS' 

C0RRECT=e'6  five.  PR6..-.495498B 

* 

UTT  2 

UTTs-DOeS  SEMrtNTlC  NETS  CET  OENTiONED  RNYLHERE" 

PEC  = ''DOES  SeriRNTIC  NETS  CET  HENTIONED  BHYIIHERE" 

C0RRECT=$/6  RVE.  PRe.«-. 5618788 

UTT  3 

UTTr”UH!CH  PfiPERS  ON  REGION  RNfiLYSlS  RLSO  DISCUSS  LRNCURCE  UNDERSTfiNDINF" 

REC  = "UH1CH  PfiPERS  ON  fl  REGION  RNfiLYSlS  SUPSYSTEtl  RNO  OESICN  rIENTION  UNDERSTAND  INC" 
C0RRECT=5/9  RVE.  PRB 6636969  ,**»***»♦ 

01 T 4 

UTT="HOU  MANY  fiRTfCLES  ON  CHESS  AND  LERRNING  RRE  THERE" 

REC="HOU  HRNY  RRTICLES  ON  CHESS  RNO  LERRN1N&  ARE  THERE" 

CORRECTsS/g  RVE.  PR6 5521664 

UTT  5 

UTT="ur'PE  INTERESTED  IN  HERRSRY" 

REC  = ''UE’RE  INTERESTED  IN  HERRSfiY" 

C0RRECT=4/4  RVE.  PRO ■>■--6638372 


Figure  2.  Harpy  results  for  the  live  demonstration,  8 September  1976. 
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TASK  Recognition  of  Ai  information  retrieval  task. 
Vocabulary  size:  1011 
Branching  factor;  3.53 
Average  fariout:  33.4 

DATA  Number  of  speakers:  1 male  speaker 


Training  set 
60 
340 
2.2 

Training  set 
747 
4049 
24.7 


for  word  hypothesizer 

sentences 

words 

minutes  of  speech 
for  word  verifier 
sentences 
words 

minutes  of  speech 


Test  set  for  all  speakers 
22  sentences 
154  words 

1.0  minute  of  speech 


PERFORMANCE  ON  THE  TEST  DATA 
SGX  word  accuracy 
73%  sentence  accuracy 
Site  semantic  accuracy 
85.0  Mipss 


Figure  3.  Hearsay-Ii  results  for  the  AI  retrieval  task  test  data. 


RESULTS  Of  LIVE  SENTENCES:  HERR3flY-JI 

UTT  1.  UTTr-I  fln  '.NTERESTEO  IN  EMCLISH" 

REC.'I  Bn  INTERESTED  IN  ENGLISH" 

UTT  2i  UTT.'BRE  rny  pbpers  bbcut  SEMRNTIC  NETUORNS" 

REC.'RRE  RNY  PAPERS  RBCUT  R SEHONTIC  NETUORN" 

UTT  3;  UTTr-OOES  SEflRNTIC  NETS  CE’  flENTIONEO  BHYUHERE' 

TIHEOUT  - 2 best  pirsas  *r»: 

too  SinUlTRNEOUS  BCI  IONS ] 

I DESIGN  BNO  SYNISX  riENTIONEO  RNYUHEREl 

UTT  4:  UTr."H0U  NANY  ARTICLES  ON  CHESS  BNO  LEBRNIMG  BRE  THERE" 
TinEOUT 

UlT  5i  UTT.“UE’RE  INTCRESTED  IN  HEBRSRY" 

REC-'UE’RE  INTERESTED  IN  HERRSBY" 


48i!  SENTENCE  BCCURRCY 
BBi  SEnflNTIC  BCCURRCY 

Figure  4.  Hearsay-11  results  for  the  live  demonstration,  8 September  1976. 
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Connected  Digit  Recognition  using  Symbolic  Representation  of  Pronunciation 
Variability  (Goodman,  Lowerre,  Reddy,  and  Scelza) 

Most  connected  speech  recognition  systems,  such  as  Harpy  and  Hearsay-ll,  use 
some  form  of  symbolic  represerttatlon  to  represent  alternative  pronunciations  of  the 
vocabulary,  whereas  most  isolated  word  recognition  systems  use  word  templates.  In 
an  attempt  to  compare  relative  performance  of  systems  that  use  symbolic 
representations  cl  words,  the  Harpy  system  was  run  on  four  tasKs  requiring  (he 
recognition  of  random  sequences  of  digits.  Recording  was  in  a computer  terminal  room 
environment  (approximately  60  dBA)  with  speakers  recording  one  session  per  day  in 
Order  to  include  as  much  intra-speaker  variability  as  possible.  Both  male  and  female 
speakers  were  used 

3-Digits  Task 

This  task  was  selected  as  a typical  numerical  data  input  task.  Sentences  are 
connected  sequences  of  three  digits,  such  as  "zero  th’'ee  eight'.  Each  of  ten  speakers 
spoke  thirty  training  sentences  and  100  test  sentences  over  a period  of  three  weeks. 
Using  speaker-specific  phoneme  templates,  the  word  error  rate  Over  all  ten  speakers 
was  about  27-. 

7-Digits  Task 

This  task,  sometimes  relered  to  as  the  "telephone  number  task”,  consists  of 
connected  seven  digit  sequences  such  as  "seven  three  nine  six  one  seven  three".  This 
tasK  was  selected  as  a benchmark.  Error  rate  for  the  single  speaker  was  \1,. 

Telephone  Input  Task 

Sentences  are  three  digit  connected  sequences,  as  in  the  3-digits  task. 
Recordings  were  taken  over  teleohone  line’-  in  order  to  determine  the  effects  of 
restricted  frequency  response,  distortion,  envelope  delay,  etc.  The  error  rale  uncier 
these  conditions  was  77 

Speaker  Independent  Task 

This  task  is  similar  to  the  3-digits  task.  However,  recognition  is  performed  using 
speaker-independent  phoneme  templates  computed  from  the  training  data  for  all 
spe.okers.  The  word  error  rate  was  about  7'/  on  test  data  of  1200  random  three-digit 
sequences  from  twenty  speakers,  including  ten  new  speakers. 


A summary  of  the  results  for  these  tasKs  is  shown  in  the  accompanying  tables. 
The  total  test  data  are  2700  sentences,  representing  more  than  an  hour  of  recorded 
speech.  While  this  is  already  a targe  amount  of  data,  a more  e>denstve  and  thorough 
study  is  to  be  initiated. 


TASK 

3-Oi gi t 

7-Digi t 

Te 1 ephone 

Speaker- 

Vocabulary  Size 

10 

10 

10 

I ndependent 

10 

Branching  Factor 

10 

10 

10 

10 

No,  of  Speakers 

10 

1 

4 

20 

fla  1 e 

7 

1 

3 

14 

F etna  1 e 

3 

1 

6 

Training  Set 

No.  of  Sentences 

300 

30 

120 

« 

300 

No.  of  Uords 

900 

210 

360 

900 

Mins,  of  Speech 

7.5 

1.4 

3.1 

7.6 

Uords/mi nuto 

120 

150 

115 

118 

Test  Set 

No.  of  Sentences 

1000 

100 

400 

1200 

No.  of  Uords 

3000 

700 

1200 

3600 

Mins,  of  Speech 

25.1 

4.8 

10.3 

33.0 

Uords/mi nute 

120 

146 

117 

109 

Performance  on  Test 
tiUord  Accuracy 

; Data 

98 

99 

93 

93 

KSent. Accuracy 

9G 

96 

82 

83 

M i pss 

3..S 

3.5 

3.5 

3.5 
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Effects  of  Branching  Factor  and  Vocabulary  Size  on  Performance  (Goodman, 
Lowerre,  and  Reddy) 

Analysis 

Analysis  of  the  languages  of  a given  set  of  recognition  tasks  permits  the 
comparison  of  the  relative  difficulties  of  the  tasks.  We  have  developed  notions  of 
equivalent  vocabulary  size,  branching  factor,  effective  branching  factor,  search  space 
size,  and  search  space  reduction  {Goodman  1976).  All  of  these  are  useful  as  relative 
comparison  measure. 

□•sign 

A family  of  languages  having  varying  characteristics  is  required  in  order  to  be 
able  to  compare  language  measures  with  actual  performance  data.  Such  a family  has 
been  generated  for  the  A!  abstract  retrieval  task  by  interactive  grammatical  inference. 
There  are  four  subfamilies  for  each  of  the  (approx.)  vocabulary  sizes  250,  500,  750, 
and  1000  words.  Several  grammars  representing  differing  branching  factors  exist 
within  each  subfamily.  With  the  250  word  grammar,  for  instance,  the  available 
branching  factors  are  1.23,  3.87,  A.6,  8.2,  8.8,  1 1.9,  33.3,  and  39.5. 

Results 

The  relationships  between  accuracy  and  speed  versus  branching  factor  and 
vcjcabulary  size  are  summarized  in  the  accompanying  tables.  As  expected,  there  is 
positive  correlation  in  all  cases.  In  the  case  of  speed  versus  branching  factor,  the 
relationship  is  almost  linear.  A more  comprehensive  study  of  measures  for  grammatical 
complexity  and  their  predictive  abilities  is  necessary  before  any  significance  can  be 
attached  to  these  preliminary  results. 


Table  1.  Effects  of  branching  factor  on  error  rates  of  the  Harpy  system  within  the  250 
word  family  of  grammars. 


Table  11. 


SIAIIG 

BRANCHING 

ERROR 

GRAfinAR 

MIPSS 

FACTOR 

RATE 

AIS0G 

G.S3 

4.G 

0% 

AIS10 

9.36 

8.2 

4% 

AIS15 

13.G5 

11.9 

6% 

A I 530 

4A.7? 

33.3 

1B% 

AISA0 

53.15 

39.5 

1B% 

Speed  versus  vocabulary 

Scze  for  H.irpy  when 

br  anching 

factor  is  held  constant  (approx.  10). 

BRANCHING 

GRAMMAR 

MIPSS 

FACTOR 

VOCABULARY  SIZE  (APPROX) 

AIS10 

9.3G 

8.2 

250 

AIMIZ 

16.77 

10.5 

5B0 

A 1X05 

2G.00 

9.5 

1000 
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APPENDICES  for  Section  III 


Appervdtx  Ill-A  fists  the  100  test  sentences  used  by  the  l-Urpy  end 
Hearsay-II  systems,  along  with  characteristics  measuring  their  complexity 
relative  to  several  grammars. 

Appendix  III-B  is  the  phonetic  dictionary  for  the  1011  words  used 
in  the  AI  retrieval  language. 

Appendix  III-C  contains  the  complete  definition  of  three  of  the 
grammars  (AIXF,  AIXi5,  and  AIX05)  used  in  testing  the  systems.  These 
grammars  have  become  standards  for  future  development  and  testing. 
AIXF  was  not  used  to  test  Harpy  because  the  network  was  too  large  to  be 
generated. 
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Appendix  I II- A.  Characteristics  of  the  Al  Retrieval  Task  sentences 


Below  is  a description  of  the  test  sentences  u4ed  for  the  Harpy  and  Hearsay-ll 
systems.  The  September  Hearsay-II  results  used  22  of  the  sentences  ranctomly 
selected  from  the  100.  The  entire  set  of  100  was  used  for  the  100  single-speaker 
test  sentences  lor  Harpy,  and  21  of  them  were  used  lor  the-  other  four  speakers 
tested  on  H?.-py. 


CMU  Test  Sentences 

The  branching  factors  previously  given  for  ihe  languages  used  by  the  CMU 
speech  understanding  systems  <HARPY  and  Mearsay-Il)  are  "static"  branching  factors 
(SBF)  (as  derived  by  Gary  Goodman  and  described  in  his  recent  thesis).  Intuitively, 
they  can  be  thought  of  as  being  derived  by  doing  a Monte  Carlo  probing  of  a network 
describing  all  acceptable  word  sequences  and  taking  the  average  of  the  number  of 
words  possible  following  any  legal  initial  sequence.  Othar  groups  have  generated 
somewhat  similar  numbers. 

What  we  present  here  is  a characterization  of  the  lexical  fanovit  allowed  by  our 
grammars  for  the  particular  test  sentences.  The  notion  is  tu  calculate  the  average 
fanout  for  each  sentence-initial  sequence  of  words  (i.e.,  going  left-to-right). 

The  method  used  here  is  the  following;  For  any  sequence  of  words,  denote  by 
Word  Branches  (WB)  the  number  of  words  that  may  legally  follow  that  sequence  in  the 
given  language.  Consider  a sentence  of  lenglli  N-1  words  to  have  N WB’s  — each  is 
calculated  from  the  initial  sequence  of  i words,  i»0,l....NI.  (I.e.,  the  first  WB  for  any 
sentence  is  always  the  same  — the  number  of  legal  first  words.)  Then,  for  any 
sentence  or  collection  of  sente''ces,  the  Average  Fanout  (AF)  is  the  arithmetic  mean  of 
the  WB’s  of  the  sentence(s). 

The  languages  used  (all  defined  using  the  same  1011-word  vocabulary)  are 
called  AIX05,  AIX15,  and  AIXF.  The  first  two  have  static  branching  factors  of  10  and 
28,  respectively.  This  summary  is  over  100  test  sentences  containing  a total  of  683 
words. 


AF 

A 1X05 

AIX15 

AIXF 

sents 

uords/sent 

33.4 

46.5 

68.0 

100 

6.83  (average  over 

17.3 

26.0 

33.4 

2 

3 

31.3 

45.4 

84.0 

10 

4 

36. 1 

50.7 

73.0 

11 

5 

29.  7 

41.5 

60.3 

21 

6 

33.6 

47.0 

70.2 

24 

7 

37.2 

51.1 

70.3 

15 

8 

30. 1 

40.5 

63.0 

9 

9 

42.3 

61.5 

70.8 

3 

10 

42.8 

57.9 

76.3 

3 

11 

21.2 

29.9 

53.4 

2 

12 
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The  100  sentences,  presented  with  farwuts  according  to  the  AIX05  language. 


[ 6G  00  6 ANY  S OF  3 THESE  3 FIENTION  132  PSYCFCLOGY  3 worris-6 
AF-33.857 

t BG  UHICH  21  COGNITIVE  1 PSYCHOLOGY  2 CONTAINED  192  UINOGRAD’S  1 
ARTICLE  1 words-G  AF-40.571 

[ GG  UHAT  2G  TOPICS  1 ARE  1 RELATED  1 TO  192  SEHANTIC  2 T^Tl/CRKS  3 . 
uords=7  AF-3G.500 

( GG  DOES  13G  PATTERN  3 DIRECTED  1 FUNCTION  1 INVOCATION  3 GET  2 
DISCUSSED  1 AN^UHERE  I worcls-g  AF-30.444 
[ GG  UHICH  21  TITLES  1 COf<lTAIN  1 TFE  1 PHRASE  192  TIME  2 COflPLEXITY  3 
uordS'=7  '^F=35.875  . 

[ GG  DOES  13G  THAT  1 ARTICLE  1 MENTION  192  TIME  2 OR  1 SPACE  1 BOIMJS 
3 uords=8  AF=51.444 

[ GG  UHICH  21  OF  2 THEM  1 DISCUSSES  192  EVALUATION  1 FUNCTIONS  3 
i_iord5=G  AF=40.8S7 

[ GG  ARE  292  THERE  2 ANY  5 ABSTRACTS  1 UHICH  1 REFER  1 TO  192  PAPERS 
1 BY  9G  NEUELL  3 uords»10  AF=G0.000 
[ GG  UHERE  5 IS  192  PREDICATE  1 CALCiiLUS  3 MENTIONED  1 word3=5 
AF=44.GG7 

[ BG  UHAT  2G  ARE  3 SOME  1 OF  1 THE  1 AREAS  1 OF  192  ARTIFICIAL  1 
INTELLIGENCE  3 uords=9  AF=29.500 
[ GE  UHAT  2G  UAS  1 I TS  1 TITLE  1 -jords=4  AF-19.000 
[ GG  UHO  5 UAS  2 THE  1 AUTHOR  i iiords=4  AF=15.000 

( GG  UHERE  5 DOES  1 HE  1 UORr:  1 uords=4  AF  = 14.800 

[ 6S  UHAT  2S  IS  4 HER  1 AFF.lIATION  1 uords=4  AF=19.G00 

{ GG  UHAT  2S  ADDRESS  1 IS  1 GIVEN  1 FOR  1 THE  1 AUTHORS  1 words-:' 

AF=12.250 

[ GG  HOU  4 MANY  8 REFERENCES  1 ARE  1 GIVEN  1 uords-S  AF-13.500 
[ GG  PLEASE  4 LIST  1 THE  1 AUTHORS  1 uords=4  AF-14.G00 

I GG  PLEASE  4 MAKE  1 ME  1 A 1 FILE  1 OF  1 THOSE  1 uords-7  AF-9.500 

( GG  CAN  2 I 1 HAVE  1 THESE  1 ABSTRACTS  1 LISTED  1 words-G  AF-10.422 

[ GG  ARE  232  ANY  S ARTICLES  2 ABOUT  192  STRUCTURED  1 PATTERN  1 

wECOCN 1 T 1 ON  3 uords=7  AF-70.375 
( GE  DO  G ANY  6 OF  3 THE  1 ABSTRACTS  1 MENTION  192  LEARNING  3 
words -=7  AF  = 34./'S0 

[ GG  HOU  4 MANY  8 OF  1 THESE  1 ALSO  1 HiXUSS  192  ABSTRACTION  3 
words=7  AF=34.500 

[ GG  UHICH  21  PAPERS  7 ON  '32  LANGUAGE  G UNDERSTANDING  4 ARE  1 ABOUT 
132  ENGLISH  3 w -rds^S  AF-S4.BG7 

[ GG  DO  G ANY  G PAPERS  S Oli  192  AUTOMATIC  7 PROGRAMMING  3 EXIST  1 
words=7  AF=35.7S0 

I GG  UHAT  2d  ABOUT  288  PROGRAM  1 VERIFICATION  3 words-A  AF-7G.800 
I GG  I 2 AM  2 INTERESTED  1 IN  132  ARTIFICIAL  1 INTELLIGENCE  3 
words=E  AF-38.143 

[ GG  THE  3 AREA  2 1 1 AM  1 INTERESTED  1 IN  1 IS  132  UICERSTANDI NG  3 
words-8  AF=30.000 

[ GG  DON’T  1 GET  1 ME  1 ANY  1 ARTICLES  1 UHICH  1 MENTION  192  GAME  2 
PLAYING  3 uords=9  AF=2B.980 

I GG  1 2 AM  2 ONLY  1 INTERESTED  1 IN  1 PAPERS  i ON  132  CHESS  4 
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words=8  AF-38.300 

[ GR  LET’S  1 RESTRICT  1 OUR  1 ATTENTION  1 tq  1 PAPERS  1 SINCE  1 
NINETEEN  1 SEVENTY  i poyn  \ Dords=10  AF-G.SBS 
C GG  DO  6 ANY  G PAPERS  5 THIS  1 YEAR  1 CITE  9G  ROSENFELD  3 words-7 
AF=23.000 

[ GG  ARE  292  COf.PUTER  7 NETUORl^S  4 n;NTIONED  2 ANYWHERE  1 word3-5 
AF-G2.000 

t GG  ARE  292  ANY  6 ARTICLES  2 ABOUT  132  GRAIN  1 CF  1 COMPUTATION  3 
words-?  AF-70.375 

[ GG  ARE  292  ANY  G ARTICLES  2 BY  96  ROSENFELO  3 words-S  AF-77.500 
[ GG  ARE  292  ANY  G BY  3G  FEIGENBAl^  3 AND  9G  FELCttlAN  1 uords-G 
AF-80. 000 

[ GG  ARE  292  THERE  2 ANY  5 ABSTRACTS  1 UHICH  1 REFER  1 TO  192  PAPERS 
1 BY  3G  HOLLAND  3 woras-10  AF-G6.000 
[ GG  ARE  292  THERE  2 ANY  5 NEU  2 PAPERS  1 ON  192  PROGRAM  i 
VERIFICATION  3 uords«8  AF-*G2.G67 
t GG  DO  G ANY  G OF  3 THESE  3 ALSO  2 MENTION  192  PLAftfJER-LIkTE  1 

Languages  3 uords=8  AF-31.333' 

I GG  DOES  136  PROBLEM  1 SOLVING  3 GET  2 MENTIONED  1 ANYUFCRE  1 
uior(Js=G  AF-38.571 

[ GG  UHICH  21  PAPERS  7 CITE  9€  NEWELL  3 AND  9S  SIMON  1 words-B 
AF=41 .429 

[ GG  ANY  1 ABSTRACTS  1 REFERRING  1 TO  192  AI  4 OR  191  ARTIFICIAL  1 
INTELLIGENCE  1 uords«8  AF=50.889 
( GG  ARE  292  ASSOCIATIVE  2 MEMORIES  4 DISCUSSED  1 IN  1 RECENT  1 
JOURNALS  1 uords=7  AF-46.000 

f GG  ARE  292  LEARNING  4 ANO  191  NEURAL  1 NETUORtCS  2 MENTIONED  2 
anywhere  i words-7  AF-G9.S7S 

C GG  DID  99  REOOY  5 PRESENT  2 A 1 PAPER  1 AT  2 IXAI  1 words-7 
AF-22. 12E 

[ 66  DIDN'T  1 THAT  1 PAPER  1 QUOTE  9G  DREYFUS  3 uords-5  AF-28.000 
[ GG  ODES  19G  PICTURE  1 RECOGNITION  3 GET  2 MENTIONED  1 ANYWHERE  1 
words=G  AF=38.57l 

[ GG  GET  1 ME  2 EVERYTHING  1 ON  192  DYNAMIC  3 CLUSTERING  3 words-S 
AF=38.286 

[ GG  GENERATE  1 A 1 COPY  1 OF  1 THOSE  1 iiords=S  AF-11.833 
[ GG  GIVE  2 ME  4 THE  1 DATE  1 OF  1 THAT  1 ABSTRACT  1 words-7 
AF=9.S25 

[ GG  HOW  4 CAN  1 I 1 USE  1 THE  1 SYSTEM  1 EFFICIENTLY  1 words-7 
AF=3.500 

[ GG  I 2 AM  2 INTERESTED  1 IN  192  LEARNING  3 words-S  AF-44.333 
[ GG  I'D  i LIKE  1 TO  2 SEE  1 THE  1 MENUS  1 words=G  AF-10.423 
[ GG  SELECT  1 FRDri  1 ARTICLES  1 ON  192  GAME  2 PLAYING  3 words-G 
A'=38. 000 

[ GG  WHAT  2S  ADDRESSES  1 ARE  1 GIVEN  1 FOR  1 THE  1 AUTHORS  1 words-7 
AF=12.2S0 

t GG  WHAT  26  PAPERS  1 ON  192  PREFERENTIAL  1 SEMANTICS  3 ARE  1 THERE  1 
words-7  AF=3G.375 

C B6  WHEN  3 WAS  134  A 20  SEMANTIC  1 NETWORK  3 LAST  2 REFERRED  1 TO  1 
words=8  AF=32.333 
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[ B6  UiHICH  2i  PAPERS  7 CiTE  35  FELDRAN  5 worus^A  Ar>3S.500 
[ 65  UHO  S HAS  1 URITTEN  1 ABOUT  192  AUTORATIC  7 PKOGRAftllNG  3 
words'G  AF-39.28G 

( 66  UHO  6 UA5  2 QUOTED  1 IN  1 THAT  1 ARTICLE  1 uords-B  AF-1 1.000 
{ 66  UHICH  21  IS  1 THE  1 OLDEST  1 uorOs-A  AF-1S.000 
( 66  ARE  292  ANY  6 NEU  1 BOOKS  1 BY  1G  TERRY  1 U! NOGRAD  3 uords-? 
AF.58.250 

[ 66  CAN  2 I 1 HAVE  1 THESE  1 ABSTRACTS  1 LISTED  1 uords-B  AF-10.429 
( 66  DID  93  CARL  1 ITEUITT  5 PRESENT  2 A 1 PAPER  1 AT  2 THE  1 IFIP  1 
REETINGS  1 IN  1 SEPTEnBER  1 uords-12  AF-14.000 
[ 66  DID  99  ANY  4 ACL  1 PAPERS  1 CITE  96  RICK  1 HAYES-ROTH  3 uorda-? 
AF-33.875 

[ 66  DO  6 ANY  6 OF  3 THOSE  1 PAPERS  1 RENTlON  192  AXIOMATIC  1 
SEMANTICS  3 uords-8  AF.31.000 

I 66  DUPING  1 UHAT  1 MONTHS  1 l^RE  1 THEY  1 PUBLISHED  1 uords-G 
AF-T 0.286 

[ 66  HOD  4 MANY  8 RECENT  1 ISSUES  1 CONCERN  192  INVARIANCE  1 FOR  1 
PROBLEM  1 SOLVING  3 uords=9  AF-27i800 
[ 66  HQU  4 MANY  8 SUMMARIES  1 DISCUSS  192  KNOULEDGE  2 BASED  1 SYSTEMS 
3 uords«7  AF-34.G2S 

f 66  HAVE  97  ANY  2 NEU  1 PAPERS  1 BY  96  LEE  1 ERMAN  3 APPEARED  1 
uords=S  AF=29.778 

I 66  I’D  1 LIKE  1 TO  2 KNOU  1 THE  1 PUBLISHERS  1 OF  1 THAT  1 STORY  1 
KOrds-S  AF=7.G00 

C 66  IS  230  HUMAN  3 BEHAVIOR  5 OR  191  HUMAN  3 MEMORY  3 DISCUSSED  2 IN 
1 A 1 RECENT  1 SUMMARY  1 uords=ll  AF-47.250 
I 66  LIST  2 THE  2 ABSTRACTS  1 BY  36  HERB  1 SIMON  3 uords-B  AF-24.429 
[ 66  UAS  230  ALLEN  2 NEUELL  3 CITED  2 IN  1 THAT  1 SUMMARY  1 words-? 
AF=45.75d 

I BG  UHAT  26  ABOUT  288  ALLEN  ? COLLINS  3 words=4  AF.77.000 
( 66  UHERE  5 DID  1 THAT  1 ARTICLE  1 APPEAR  1 uords=5  AF-12.500 
[ 66  UHO  5 HAS  1 URITTEN  1 ABOUl  192  LANGUAGE  6 COMPREHENSION  3 AW 
131  LANGUAGE  6 DESIGN  . Mords=9  A(^=47.200 
[ 66  QUIT  1 LISTING  1 PLEASE  1 -Jv  •ds=3  AF=17.250 

I 66  UEREN’T  1 SOME  1 ARTICLES  1 PUBLISHED  1 ON  192  GOAL  1 SEEKING  1 
COMPONENTS  3 uords=8  AF=29.6G7 

r 66  UHAT  26  SORTS  1 OF  192  LANGUAGE  6 PRIMITIVES  3 ARE  1 URITTEN  1 
UP  1 uords=3  AF=33.800 

( 66  HASN’T  132  A 21  CURRENT  1 REPORT  1 ON  192  PRODUCTION  1 SYSTEMS  3 
BEEN  1 RELEASED  1 uords-9  AF=47.900 
[ BG  ARE  292  THERE  2 ANY  5 ISSUES  1 ABOUT  192  COOPERATING  1 SOURCES  1 
OF  1 KNOULEOGE  3 uords=9  AP=5B.400 
( 66  DID  93  ViC  1 LESSER  5 PRESENT  2 PAPERS  1 AT  2 IFIP  1 words-7 
AF=22.12S 

( 66  DID  33  ANYONE  1 PUBLISH  1 ABOUT  132  LARGE  1 DATA  1 BASES  3 IN  1 
COMMUNICATIONS  1 OF  1 THE  1 ACM  1 words=12  AF-28.385 
( 66  DO  6 ANY  6 AUTHORS  1 DESCRIBE  132  DRAGON  3 uords-S  AF-45.G67 
I 66  DOES  136  HE  1 UQRK  I AT  1 CMU  1 uords-S  AF.44.333 
( 66  DO  6 ANY  6 RECENT  4 ACM  1 CONFERENCES  1 CONSIDER  192  SEMANTIC  2 
NETS  3 OR  191  SEMANTIC  2 NETUORKS  1 uords  = ll  AF-SS.SSJj 
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C G6  DO  G RESPOf^ES  1 EVER  1 COME  1 PASTER  1 ^ords-S  AF.12.66V 
( G6  HAS  96  LEE  1 ERflAN  A BEEN  1 REFERENCED  1 IN  1 ANY  1 OF  1 THOSE  1 
words«9  AF-17.3B8 

( GG  HAS  3G  ALLEN  2 NEUELL  A PUBLISHED  2 ANYTHING  I RECENTLY  1 
uorclSaG  Ar»2A.S71 

[ GG  HAVE  97  ANY  2 NEU  1 PARERS  1 BY  3h  TERRY  1 UINUGRAD  3 APPEARED  1 
words*=8  AF=23.77S 

I GG  HCU  A BIG  1 IS  1 THE  1 DATA  1 BASE  1 words-6  AF  = 10.71A 
t GG  HOU  A MANY  8 OF  1 THESE  1 ALSO  1 DISCUSS  132  DYNAJIIC  3 BINDING  3 
words=8  AF=3i.090 

[ GG  HOU  A MANY  8 RECENT  I ISSL£S  1 CONCERN  132  DISPLAY  1 TERfI INALS  3 
uords=7  AF-3A.500 

[ GG  KILL  1 THE  1 LISTING  1 uords-3  AF.17.250 

I GG  PLEASE  A MAKE  1 ME  1 A i FILE  1 Of  1 THOSE  1 uords-7  AF-9.500 
[ GG  UHAT  2G  IS  A HIS  1 AFFILIATION  1 uords.A  AF-19.G00 
[ 66  UHICH  21  OF  2 THESE  5 CITES  96  PERRY  1 THORNOYKE  3 uords=B 
AF=27.71A 

I GG  UHICH  21  PAPERS  7 ON  192  DESIGN  G IN  I THE  1 ARTS  A ALSO  2 
DISCUSS  192  DESIGN  5 AUTOflATION  3 words  = ll  AF-Al.GG? 

I GG  UHO  5 UAS  2 QUOTED  1 IN  1 THAT  1 ARTICLE  1 woros=G  AF-11.060 
t GG  WHICH  21  PAPERS  7 WERE  1 WRITTEN  2 AT  1 NRL  1 OR  1 AT  1 SHC  1 
i-iords=9  AF=10.200 
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(-.O'  (EVL.O)  EVC  (EYB.O)  3 IH?  (►  '.-.0).-14))  (((,0)  L UH2  (.-  (-,0),-(4})  (K.0) 

(-.0)  AE4  (_  (-.0).-,0'  (D.Q)  AES  (.-  (-.0).-)  T (EVL.O)  EYO  (EYR,0)  SH  iHS  N 
(-.0)  (IX.UH)  (►  (-.0).-)  (0,0)  AE  (♦-  (-.0).-)  T IX  V (F.O) 

(-.0)  IK3  U-  (-.0).-)  (D.O).OX)  IH?  SH  IHS  N 
(-.0)  ((AE.IXL.UH)  (-  (-,0),-)  08  8 EH2'  S (HH,0> 

(-.O)  ((AE.IX).ljH)  (-.0).  ) D8  R EM2'  S 1M4  (2[41,(2,0)  3) 

(-.0)  (IX.UH)  (►  (-.0).-)  (0,0)  V (AVI,0)  AVC  (AVR,0)  (214), (2,0)  S)  (1H3.IV>  NX 
(-,0)  AX  S TH  EM  <_  (-.C>.-)  T IH  (-  (-.0),-)  S (HHO) 

(-,0)  (EH3AH)  F (IH.O)  EL  lY  (EVL.O)  EVD  (EYB.0'  SH  IHS  N 

(-,0)  AX  ( 0)  F EH2  I IV2  (EVl.O)  EVC  (EVR.O)  SH  IHS  N (2(4j,(2,0)  S) 

(-.0)  AE'  F ((»-  (-.0).-)  T.DX)  ER 

(-.0)  (EVL.O)  EVC  (EVR,0>  (AVl.O)  AVC  {AVfi.O) 

(-.0)  AE3  El  (V  (-.0).-)  SH;,8!  IH  (-,0).-t4})  (B.O)  B (EVl.O)  EVC  (EYB.O)  IH2  (►  (-,0>,-[4))  (X,0) 

(-.0)  AE4  EL  (-  (-.0).-)  (G.O)  0W3  E13 

(-.0)  AE  EL  <4-  (-.0),-)  (G.O)  (AA,OW)  R'  IH  (TKDH)  (1H,1X0>  M 

(-.0)  AE  EL  (-  (-.0).-)  (G.O)  (AA,0W)  B'  IH  (THWO  M IH  (.,0)..I4})  (K.0) 

(-,0:  0W4'  EL 

(-.0)  0W4i  EL  (-.0)  (AA4.0)  ERcl?  14)  (-.0)  N UH  (N.DX) 

(-.0)  AE'  EL3  (IH6  N.EN) 

(-.01  (AO  OWfl)  EL  S (IH6  {0W2.0).0W) 

(-.0)  AO  EL  W (EYL.O)  EVC  (EYB.O)  (Z|41.(2.0)  S) 

(-.0)  ((EH2'  AE3')  M.EM') 

(-.0'  dX'.AX')  M UH2  NX 
(-.0)  AES'  lEMN) 

(-.0)  AES  (N.EN)  A£4'  (EL.L)  0W4  (-  (-.0).-)  SH'.B}  TV 
(-.0)  UHfl  N AE  ELS'  (UH2;2.6).1H6.0)  S IHfi  S (HM.O) 

( .0)  AES  N EL2  (AVL.O)  AYC  (AYR,D)  (2|4).(Z,0)  S)  EB2 
(-.0)  AES'lS.lO)  N (-  (-.0).-.0)  (0,0) 

(-,0)  AEdi  (N  OX) 

(-.0)  AH  N AA2I  (CH.TH)  (tR.AA2) 

I-  0)  AES'  N S ER 

( ,0)  AEfl  N S'  (R.EB)  IHS  NX 

(-.01  AE4  (N.-)  (-  (-.0).-.0)  TH  I)«J'  N lY 

(-.0)  ([H3.tH)  (N;2;.DX)  IV-  (IY3,0) 

(-.0)  (LHaEH)  (N:2J.0X)  IV'  (IV3.0)  (-.0)  W ah  (N,DX) 

(-.0)  (EH3EH)  (N;2;,0X)  iV  TH  (!H3.iY)  NX 
(-.0)  iEH3EH)  (N:2I.0X)  IY'  (-.0)  W (EH3.0)  ER 
(-.0)  iAH3UH2)  (-  (-.0).-)  (P.PH)  IY2'  ERI.IS) 

(-.0)  (AHaUH2)  (-.0),-)  (P.PH)  1Y2'  ERI.IS;  (^  (-.0),-)  (D,DH,0) 

(-.0)  AE3  (-  (-,0),-)  (P  l.Pl  (L.O))  IH6  <•-  (-.0),-)  (K,0)  (EYl.O)  EVC  (EYR.O)  SH  1H5  N 
(-.0)  EH3  (-.0),-)  (P  R PR  (R.0)>  EH2'  N (-.0),-,0)  T IH4  S (HH.O) 

(-.O'  UH;z'  c-  (-.0).-)  <P  R.PB  (R.O)  OWZ'  (►  (-.0),-)  SHj.S) 

(-.O)  itYl.O)  EVC  (EVR.O)  (.-  (-.0).-)  (P  R.PR  (R.O))  (IH  EL,EL2,' 

\-,0)  AA  fi  (..  (.,0),-14!)  (8.0)  IV  (►  (-.0).-)  (B.O) 
f-.Oi  (AAail!  (EB2.ER).ER2') 

(-.0)  IH2'  FR  1V2  UH 
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OlcVcfV»rv 


ACiFAS 
ARf  N'l 
ARPA 
APT 

ARTICCF 

ARTICLFS 

ARTinCIAl 

ARTS 

ASIMOV 

ASK 

ASSI  STIII  V 

ASSERTIONS 

AGSI.VIEATION 

ASSOCIATION 

ASSOCIATIVE 

AT 

ATTENTION 

AUGMENTEO 

AUGUST 

AUTHOR 

AUTitOfiS 

AUTOMATCO 

automatic 

AL'TOM.ATION 

AVAUASlt 

AWASU 

AXIOMATIC 

AXIOMS 

AZRIEL 

Q.>r.KGi\KIMON 

BANtk’jl 

bank 

BARROW 

BASE 

BASfBAEl 

BASED 

BASES 

RATES 

BAV 

SEE!.' 

srrnzr 

Ell  IIAViOR 
S'  i !EF 
IiER'  ELl  V 
cr  R{  1 i\i  w 
fir  RK'ARI) 

BERT 
SE’  WEi  !V 
i i I f ■ 
lilH 

BINDING 

lllN’DINGS 

UIOMIDIClNf 

Ui  I USOE 

Sl.OCK 

liOCPOW 

iiONNIC 

HOOK 


i-,0?  IH7'  ER  r/2  UH  (7!4;.(Z01  S) 

(-,0)  <AA;t(  (ERJ.ffci.ERZ'l  (N  <.-  <-.0).-I<i;)  (T.OI.OXI 
(-.0)  AA'  <E».f  R7)  (.-  P:1  j III 

(-.0)  AA:  (ER.Ei)  i..0>.-i4il  (T.Ol.DX) 

<-.OI  (AAS  (ER.O).  tR2)  ((-  (-,0).-)  T.OXI  1H8'  (•-  (-.0).-)  K(  I J EtZ 

t-.O)  (AAS  (ER.O>.  ER2)  ((.-  <.,0>.-)  T.OX)  IHB'  K(  1 1 EL  CZ(4;,(Z.O)  S> 

(-.01  (AAU  <£R.O).  EH7>  ({..  (-.0).-)  T.OX)  IHT'  (-.O)  f IH3  SM  EL 

(-.0)  AA3'  (tR.O)  (-  S (HH.O) 

t-.O)  AE  (ZIA’.iZ.Oy  S;  i“6  t.  0W4'  F (HH,0) 

t-.O)  AF:t  S - (K',0) 

t-.O)  UH4  S ElPl  M-  (-,0).-(«:)  (fl.O)  L IV 

(-.0)  All  s EW  Sit  lii'i  N (2;4;.(Z.O)  S) 

f-,C)  IHu  S 111  M EEJ  <IVL,0)  EVC  tEYR.O)  SH  IrtS  N 

(-.0)  UMA  S own  (S.SH)  iV  AC'  3H  (AX.1X  IH5>  tN.DX) 

t-.O)  UIP!'  G 0W3  (5.SH>  IV  1H7  ((►  (-.0),-)  T.OXT  IN3  V (F.O) 

(-.0)  tAEA'.AC?'!  ((.  (..C).-J4;)  (T,0>,DX) 

<-,0)  (1112  IIH)  (-.0).-)  T EH2I  N 5ll  11(5  N 

(-.0)  AA  (.-  (-.0).-;  M EH  N «►  (-,0).-.0)  T.OX)  1H3  (►  I-.O).-)  (0.0) 

t-,0)  AO  (-.0).-)  iG.O)  1H3'  S -14;  (T.O) 

(-.0)  AO'  (-.0)  TH  ER 

(-.0)  AO'  (-.0)  Th  ER  (f  R3.0)  tZlAI.lZ.O)  S) 

(-.0)  AO  (i-  (-.0).-)  T.OX)  EH3  M (EYL.O)  EVC  (EVR.O)  tt^  (-.0),-)  T.OX)  IH3  (.-  (-,0),-;  (0,0) 
t-.O)  AO  ((.-  (-.0).-)  T.OX)  EH3  M AT'  ((.-  (-.0).->  T.DX'T  IH3  (►  t-.Ol.-lAJ)  (X.O) 

(-.0)  AA  (-.0)  -'  T.OX,  0W2'  M EEVl.Oi  EVC  (EVfi.O)  SH  1H5  N 
(-.0)  All  V tt'l.  lEYI.O!  EVC  £VH)  El'  (IX,IH)  (.-  (-.01.-[4!)  (B,0)  EL 
(-.0,  UW7  W UV/4  ER  !(►  (-.0).-)  tO.O).DX) 

(-.0)  AE  (-.0).-)  S IV  UK  M AE  ((►  (-.0).-)  T,DX>  IH3  U (-.0),-)  (tC,0) 

(-.0!  AF  (►  (-,0).-)  s iV  UH  M (Z:4;,(Z.O)  Si 

(-.0)  AC5'  (Z|4;.tZ.<'l  S)  IH  (E».«>  fl2 

(►  IA;)  (BO)  AC4  (►  (-.0),  ) (0,0)  AE4'  M 1H6  (N.OX) 

t-.o:  -;4;i  tB.o,  ;k7  n eR'  (►  (-.o).-)  sh;,b;-  iv 

(.-  (.,o),-;4;)  (H.o)  AE'  NX  (-,0).-i4;)  (k.(j) 

(-.o>.-,’4:>  (9.0)  eh3'  'Eh.R)  ow 

(.-  (-,0>,-;4;>  (fl.U)  (EVL.C)  EVC  (EVR.O)  S (HH.O! 

(-  t-.O).-;*;)  (9,0)  (EVi.O  EYC  '£VP,0>  S ■ tB.O)  AO  EL2 
(»-  (-.o),-.'.);)  (B.n-  lEVi.o)  EVC  (EVR.O)  S -|4;  a.o) 

(-  '-.o).-:';;)  (B.o)  (f  vi.o)  evc  (Evb.o)  s Ih4  (Z:4;,(Z,0)  S) 

(-  1 .CD.-.au  IR.O)  (?Vi..O)  fvc  (EVR.O)  (.•  <.,0).-)  S (HH.O) 
i-.oi.-i':;)  iB.O)  tfvL.O!  EvC'  (Eyr.o) 

(r-  l-.0).-I4I)  iB.O)  iH2'  'N.DXI 

(►  (-.o),-;4:)  (B.n,  uv.Uii  f , ao  eh 

(*-  (-.0,,  :4;)  (Hi))  W (i•l■.llll?,0)  tfVl.O)  EVC  (EVR.O)  V V EH3 
(-  (-,0>.  ;4:)  (H.O)  (iii.lY)  L iV  F (HH.O) 
i>-  ,-.C>  up;,  (H.O)  FR7  (-  ( .O,.-/.;,  (K.O)  L'  [V2 
(-.C'*-.'P;)  (HO)  cR.'  I-  jKP  r;  (Eho>  eh 
(-  0)  (H.O)  'AA'T.O;  EW7  K AA3'  EH  i-  (-.0),-)  (0,0) 

(.-  (-.0).-:');,  (S  O)  iR.)'  (-.o;.-;4’,  o.O'.dx) 

(-.o).-:-',:,  (SO)  'iii.iY'  (-.0).-)  T'  ow  ik2  (n.ox) 

<-  (-.01. (H.O)  'iH'.i.i-;  i-  i-.o'.-)  (GO) 

(.-  (-.0,  -ip;,  (S.o)  III-  Li 

(»-  (-.0),-;p;'  (H.o>  iavi.o,  avc  (Ayr.o)  n (-  (-,o),-,0)  (D.o)  (iH3,iv)  nx 
(.-  (-.r.;,-,p;-,  ■ti.(')  (AVi.oi  AVC  (Avs.o)  n (►  (-,oi,-.D)  (0,0;  Chs.IY)  nx  (Zipi.iZ.O)  Sj 
(-  (-.0).-;P;)  (SO)  (AVl.O)  avc  (AVR.O,  ow  M EH2'  ((.-  (-.0).-)  (O.OI.DX)  IH4  S IH6  (N.OX) 
(-,0),-:P:)  (H.O)  L AH?'  (-.0).-)  s ow 
I-  (-.0),-,P;,  (ti.O)  L AO'  (-  (..O',, -IP;)  (K.O, 

(-  (-.o;.-:p;)  (SO)-  -.awl.o-  awc  (awr.o  (-.oi.-iP))  (B.O)  9 Ow 

(.-  (-.0)  -IP!)  (H.O)  AA'  .N  iV 

(f  (-.c:.-;');)  (B.O)  liWUP'  (.-  (-.oi  .;4;)  ix  ni 
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Dictiontry 


BCXJKS 

BOUNDS 

brain 

BRUCE 

BUCHANAN 

BUSINESS 

BUT 

BY 

CACM 

CAI 

CALCULUS 

CAN 

CAPABILITirS 


(.-  (-.O.'.-IA))  (B.O)  UW'[4;  (*.  (-,0).-)  S (HH.O) 

(.-  (-.0),-;4!)  (B.n)  <AWL,0)  AWC  (AWR.O)  N (h  (-.0),-)  S (HH,0) 
(-  (B.n)  r <tvL.oj  evc  <£vr,0)  (n.oxi 

(>-  (0,0)  R UW3'  S (HH,0> 

(-,0).-|4!)  (8,0)  V 1V21  (►  (-,0),->  (K,0>  AES  N UH4  IN.DX) 

(.-  (-,0).-{4})  (B,0)  IH  (Z(4J.(Z.O)  S)  (N  IX, EN)  S (HH,0> 

(-  (-,0>.-!‘J:)  (0,0)  UH>  «►  (-,0),-(4J)  (T,0>,DK) 

(^  (-,0>,-|4;)  (0,0)  (AVl,0)  AVC‘  (AVR,n) 

(-,0)  S IV  (EVL.O)  EVC  (£VR,0>  S IV  AH2  M 

(-.0)  S IV  (EVL.O)  EVC  (EVR.O)  (AYL.O)  AVC  (AVR,C> 

(^  (-.0),-)  (K,0>  A£3>  EL4  (►  (-,0),-)  (K.O)  IH3  L IH6  S (HH.0) 

(-  (-,0),-)  (K,r.)  (AE ALIMS')  (N,OX) 


(•-  (-.0),-)  (K.C)  (EYL,0)  EVC  (EVB.O)  ;V  (-.0),-)  P 0W3  (-  (-,0),-!4})  (8,0)  IH  EL  IH6 
CAR  (►  <-,0),-)  (X,(l)  AA'  (E«2;,i2),0)  ER 

CARL  (-.0),-)  (K,0)  AA3I  EH2  ELS 


(-.0),-)  T IV  (Z(4},(Z,0)  S) 


CARTOCRAPHV  (^  (-. 

CASE  (^  (- 

CAUSAL  (-  (-,' 

CEASE  ( ,0) 

CELl  (-.0) 

CHAHNIAK  (►  (-, 

CHECKER  (>-  (-,1 

CHECKING  (- 

CHLSS  (^  (- 

CHOOSE  (- 

CHRISTOPHER  (-. 

CHUCK  (.-  (- 

CIRCLE  (-.0) 

CIRCUIT  (-,0) 

CIRCUITS  (-,0) 

CITE  (.,0) 

CITED  (.,0) 

CITES  (-,0) 

CLIMBING  (.-  (-,' 

CLUSTERING  (^  (-,' 

CMIJ  (-,0) 

CODE  (- 

CODING  (k  (■ 

COGNITION  (-  (-. 

COGNITIVE  (-  (. 

COLOY  (-  (-, 

COLES  (v  ( 

COLLINS  {<-  {-.• 

COME  (-,' 

COMMENTS  {<-  (-.' 

COMM'TTEE 
COMMON  (>-  ( 

COMMI1MCATION(^  (-, 
COMMUNICATIONS!^  ( 
COMPLEX  (^  (-, 

COMI’i  EXITV  (►  (-. 

COMI'ONI ' ■'S  (-  (-. 

COMPI)ri(LNS10N(^  (-. 
COMPUTATION  (^  (-. 

COMPUTATIONAL  (►  ( 

COMPUTER  (►  (-, 

COMPUTERS  (-  (-. 


0),-)  (K,0)  AA3'  EH  (.•  (-.0),-)  T AO  (►  (-,0),-)  (G,0>  ER  F IV 
0>,-)  (K,0)  (fVlO)  EVC  (EVR.O)  S (HH.O) 

0),-)  (K.O)  AO'  (2;n;.(Z.O)  S)  uh2  El 
S IV'  S (HH.O) 

S (EH.AAS)  rL2' 

0),-)  shi.r;  eh  n IV'  AES  (-,0),-i4;)  (k,o> 

0).-)  SH|.10)  EH'  (^  (-.0).-)  (K.O)  ER 
0),-)  SM  £•(  (-  (-.0)..)  (K.O)  0113, IV)  NX 
0).-)  SH(,I01  EH/l'  5 (HH.O) 

0).-)  SH  (1H2,0>  IIH'  (Z14;.(.Z.0>  S) 

0).-)  (K.O)  R IH2  S'  - T IH  (-.0)  F EH2 
,0),-)  SH'.lOj  AAS*  («-  (-.O.-IA))  (K.O) 

S (IH,1X.0)  EH  (-  (-.0).-)  (K,0)  EL 
S (IH.O)  ER  (-  (-.0),-)  (K,0>  UH*  ((►  (-.0),-;4!)  (T.OI.DX) 

S ER  (<■  (-.0),-)  (K.O)  IX  (►  (-.0),-)  S (HH.O; 

S (AVL.C)  AVC!  (AVR.O)  ((♦•  (-.0),-l4i)  (V,0),DX> 

S (AVl.O)  AVC  (AVR.O)  <(^  (-.0),-)  T,DX)  IH3  (.-  (-.O),-)  (0,0) 

S (AVL.O)  AVC  (AVR.O)  (>-  (-.C),-)  S (HH.O) 

0),-)  (K.O)  I (AYL.O)  AVC  (AVR.O)  M IH5  NX 
01,-)  (K.O)  I UH2I  S - T ER  <IH3,IV)  NX 
S IV  £H2  M'  V (IHfl.O)  UW2 
,0),-)  (K.O)  OW  <-  (-,0).-)  (D.O) 

0),-)  (K.O)  OW  (^  (-.0).-)  (O.OX)  (IM3,IY)  NX 

0),-)  (K.O)  AA3  (-.0).-)  (G.O)  N 1H3'  SH  1H5  N 

0).-)  (K.O)  AA3'  ((►  (-.0),-.0)  N.DX)  IHR  ((^  (-.0),-)  T.DX)  IHfl  V (F.O) 

0).-  / (K.O)  0W3'  EL 3 (^  (.,0),-!4;)  (8,0)  IV 
0),-)  (K.O)  OW  EL  (Z;4:,(2,0)  S) 

0),-(  (K.O)  AO  ELS  UH2  N (ZJA', (Z.O)  S) 

0).  > iK.O)  AA5'  M 

,0),-)  (K.O)  M M EH2'  N (.-  (-.0),-)  5 (HH.O) 

01.-)  (K.O)  EM'  )H2  {(<-  (-.0),-)  T.DX)  lY 
,0),-)  iK  O)  AA  M AX  (N.OX) 

,0),-)  (K.O)  AH  M UV/S  N IMS  (v  (-.0),-)  (K.O)  (EYL.O)  EVC  (EVR.O)  SH  f - 
,0),-)  (K,0)  AH  M UW  N IH  (►  (-.0).-)  (K.O)  (EVL.O)  EVC  (EVR.O)  SH  IH5  N . ,4),(Z,0)  S) 
,0),-)  (K.O)  AA  M (^  (-.0).-.0)  (P  L.PL  (1,0))  EH  (►  (-,0),-)  S (HH,0> 

,0),-)  (K,0)  AA12)  M (-.0).  ,0)  (P  L.Pl  (L.O))  EHfl'  (-,0).-)  S IH7  ((►  (-.0),-)  T.DX)  IV 

0),-)  (K.O)  AX  M (-.0).-.0)  P OW  N AX  N (-  (-.0),-)  S (HH.O) 

01,-)  (K.O)  AA  M (-  (-.0),-.0)  (P  R.PR  (H.O))  lY  HP2!  EH2  N SH  Ii^k  n 

01,-)  (K.O)  AA'  M (^  P IH3  (-  (-.0).-)  T (EVL.O)  EVC  SH  IMS  N 

0!.-1  >K,())  AA  M (-.0).-.0)  P 1M3  (-  (-.0),-)  T (EVL.O)  EVC  (ES  ..,0)  SH  1H5  N (AH  EL.EL) 

01,-)  (K.O)  (AM2  M EM)  (-.0),-,0)  (P.PH)  IH?'  (UW2.0)  <(*-  (-.0),-)  T.OX)  ER 

01,-)  (K.O)  (Aii2  M.iM)  (-  (-,0),-.0)  (P.PM)  |H2i  (UW3,0)  (1-  (-.0),-)  T.OX)  ER  (Z(4:.(Z,0)  S) 
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Oicliour  y 


COMl'ilTIIv'G 

COtJCEPTUAL 

COr.'CeWN 

CONCEHN'ED 

CONCtWN'lNG 

COtJCURHClJT 

COKTKHtKCt 

COf.TI  MCK'CES 

COKTINf 

COI^SIOER 

CONSIDEPCI) 

CONSl  PAINT 

CONSTPUCTIK'G 

CONST  RUCTION 

CONSULTANT 

CONSULTATION 

CONSULTATIONS 

CONTAIN 

CONTAINED 

CONTAINS 

CONTEN'T 

CONTINUOUS 

control 

CONTN’OLLrn 

CONVENTION 

CONVENflCNS 

COOPCWAT II, 'G 

COOPl.WAI  ION 

CCPV 

COPYING 

CORHI  CTNrr.S 

COi,i  u 

ClJWPI  NT 

CURVED 

CVW.DN'TICS 

CVCi  1C 

iTANNV 

DATA 

1)AT( 

OATES 

DAVi, 

DAVID 

OEIIAIE 

DEOEMHER 

DECISION 

DEOUCUON 

DEDUCTIVE 

DEMAND 

DENOTATIONAL 

DEPTH 

DERIVATION 

OESCRlDt 

DESCRIPTION 

DESCRIPTIONS 

DESIGN 
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(,.  (-,0,.-)  iN.O)  <A:i2  W.rW)  <-  i-,C),-,0)  (P.PID  IA2'-.  CUV,'3,0;  ((,-  (-,0),-)  T,0X)  IH2  NX 
<-,0i,-)  IK.O)  AX  N S Eri  (-.0).-)  SHj.iOl  UW  EL 
(-,0),-)  (N.o;  (iH7.lH3l  N S ER'  (N.OX) 

(_  (-,0),-)  'K,0)  (iH7,:H3)  N S ER'  N i-  (-,0),-)  (0,0) 

<-.C),-)  (K.O)  (IH7.IH3)  N S ERI  N IH5  NX 

(-,0),-)  (K.0)  IHS  N (-.0).-)  Ki  ER  EH2  N ((,-  (-.0),-i4J)  (T,0),DK) 

<-,o),-)  (*:,o)  AA'  N (-.0)  f £R  me  n s (hh,o) 

(-.0),-)  (K.O)  AAI  N (-.0)  F Efi  IHC  N S IH4  S (hH,0) 

<-,0>.-)  (K.O)  (il(7.IH31  N FI  (AYL.O)  AVC  (AVS.O)  (N.DX) 

(„  (.,0),-  > IM3  N S IU3I  ((►  (-,0),-)  (0,0), DX)  £R 

(,-  (-,0),-)  (K.O)  ({IH7.1H3)  N,fN)  S :H2i  ((,-  (-.0).-)  (DO),DX)  EH  (,-  (-.0),-)  (0,0) 

(_  (-.01.-)  (K.O)  <11(7, IH3)  N S - DR  R (EVL.O)  EVC  (EVR.O)  N (O.Ti 
(,-  (-.0).-T  sK.O)  (IH7,IH3)  N S - OR  R EHS'  (-  (-.0),-)  T (IHJ.IY)  NX 

(,-  ( .0>.-)  (K.O)  (1H7.IH3)  N S - DR  R EH3I  i-  (-.0),-)  5H  IMS  N 

(,-  (-.0).-)  (K.O)  1H7I  N S AO  EL  i-  (-,0),-)  T IH6  N (h  (-.0),-)  (O.T) 

(,-  (-.0),-T  (K.O)  AA  N S EL3  (-,0),-)  T (EVL.O)  EVC  (EVR.O)  SH  IH5  N 

(>-  (-.0).-)  (K.O)  AA  N S AX  (L,0)  (-  (-,0).-)  T (EVL.O)  EVC  (EVR.O)  SH  IHS  N (Zi4;,(Z,0)  S) 

{>-  ( ,0),-)  (K.O)  (;U7,iH;0  N ((^  <-,0),-)  T.DX)  ,rVL,0)  EVC'  (EVR.O)  (N.iDX) 

(-  (-.0).-)  'K.O)  <)K7.IH3)  N ((►  (-.0).-)  T.DX)  (EVL.O)  EVCi  (EVR.O)  N (-  (-.0),-)  (D.DKO) 

(_  (-.0),-)  (K.O)  (IH7.IH3)  N ((,-  (-.0).-)  T.DX)  (fVL.O)  EVC'  (EVR.O)  N (2141,(Z,0)  S) 

(_  (-,0).->  (K.O)  AA  N (-.0).-)  T EhO'  (-.0).-)  S -14;  (T.O; 

(,-  (-.01.-1  (K.O)  IH3I  N (.■  (-.0).-)  T il(3  N Y UH2  AH  S (HH.O) 

(„  (-.0),-)  (K.O)  (1!</,IH3)  N (i.  (-.0),-)  (SHl.lCj.T)  R OW  EL 

(,-  (-,01,.  ( ,<,())  ()i(7,:h;d  n (.-  ( ,0).-)  (T.shi.iod  r ow  ‘Ei.o)  (-,o),-)  (O.o) 

(.-  (-.01,-1  (K  '.I)  ()H7.iH:D  N (V.D  IIH  N (.■  (.-,0)  -)  SH  IHS  N 

(<-  (-.Oo-  I (K.O)  (!I)7.IH3)  N (V,f)  IIH’  N (<•  (.-.O)  -)  Sri  Ii'5  N (Z;4;,(Z,0)  S) 

(-  (-.0),-)  iK.O;  OW  AC’  (-.0).-)  (P.O)  fW2  (!  VL.O)  £ VC  (EVR.O)  ((^  (-.O),-)  T.DX)  (1H3,IV)  NX 

(-  <-,01.-1  (K.O)  OW  AO'  (-  (•  ,('),-)  (P.O)  ER2  (EVl.O)  EVC  (EVR.O)  SH  IH5  N 

(<•  (-.0),-)  (K.O)  AA’  (f  ( -.O',-)  P IV 

(<-  <-,0',-)  (K.O)  AO'  (.-  (-,C).-)  P iV  I'lriS.O;  NX 

O-  ( ,01,-)  iK.O/  i«  EH  ((.-  (-.0).-)  T.DX)  N iX  S PiH.O) 

(<-  '-,0).-'  (K.O)  UH'  (-.0),-)  (0.0) 

(*-  ( ,0),-)  (K.O)  («'  tins  N.tN.'  (,V  (..0).-;4,)  (T,0>.DX) 

(-  ( ,01,  ' (K.O)  i.iliAi  Eli  (I  .V)  (v  (-,C>.-)  (O.O.r 

(•,0)  S (AVl.O)  AVC  (AVRO)  (,-  (-.O',-/,;)  <B,0)  LR  N bil  ((,-  (-.0),-)  T.DX)  jH3  (k  (-.O),-)  S (HH.O) 
(-.0)  S 1H3'  (^  (-.0).-;>T;)  (K.O;  L UH2  (,-  {-,0),-:4;)  (K.O) 

<-,(D,-)  (D.O),DX)  AE4'  N lY 

((,-  (-,(1),-)  (00), DX)  (EVL.O)  EVC  (EVR.O)  ((-  (-,0,.-i  i0,0),DX;  LIH 
((,-  (-,0),-)  lO.Oj.DX)  (LVi.O)  EVC  (EVR.O)  ((-  ,-,0).-!4;)  (T,0),DX! 

((^  (-.0),-)  (;)<)).r;X'  ifvi.o)  eyc  (E'^ro'  i-.c s (hh.o 

((,-  (-.O).- ) iPO).DX)  (f  Vi.o)  EVC  (EVR.O;  V a (.1 

((_  (-.0),-)  ■•;).n),DX)  (EVL.O)  EVC-  (EYR.O;  V ;■..•)  iv-  I .0'.-)  (D.O) 

((,-  (-.fr),-)  fOO).DX)  IH2  {-  (-.O),-:-:;)  (B.O)  lETV.O)  EVC  (EVR.O)  ((,-  (-.O.-IAj)  (T.O.O/) 

((_  (-.0).-)  (()0),DX)  IH?  S EH'  M (^  (-.0),-/:)  lE)  0)  ER 
((,-  (■.O',-)  lO.OXDX)  !i2  S IH  !-,0),-)  SH  IH'j  N 

((,-  (-,0),-)  (O.O.'.DXI  (;V.lii2,IXi  (<-  (-,C).->  (O.oi  AA3  i-,0),-)  Sh  IHS  N 

(-.o).->  ,().()),D>:)  (Iv.;h2.ix)  <-,o)-i  D.ox)  aa?'  (-.o.  ) T.oxi  imp  v (F.o) 

<-,0).-)  '.0()),DX,  ill'  M AC4  N (,-  (-.0).-)  (DO) 

((_  (.,(').-)  (()0),DX)  IV  N OW  <-  <-,0),-)  T (EVl.O)  EVC  (EVR.O)  SH  IHS  N EL 
((>-  (-.0).-)  (O.TD.DX)  CH4I  I,-  ( ,0).-)  TH  (HH.O/ 

((,-  (-.0).- 1 ((>.0).DX)  iHP  ER  <iM.0>  V (EVl.O)  EVC  (EVR.O)  SH'  IHb  N 

((,-  (-.0),-,'  %i).()),DX)  IHA-  S - (K.WH.O)  R (AVL.O;  AVC  (AVR.O)  B 

((^  (-,()).-)  '.O.OI.DX)  IhA  S - (K.O)  R ;h2  (-  (-.0  -)  SH  ills  N 

<(,-  ( ,0;.-)  (O.O.'.DX)  iX  S - (K.O)  S IH  (,-  (-.0),-'  SH  IHS  N (Z/I  'Z.O)  S) 

(O.O'.DX)  (iHC.lX.O,  (Zi4;,(Z.O)  S)  (AVL.O)  AYC  (AVR.O)  (N.OX) 

((*■  (-.0),-)  <D.O).DX)  (IM.IY)  (Z;4;,(Z.O)  S)  (AV,,0)  AVC  (AVR  CD  LR 

((,-  (-.0).-)  <0.0>.DX)  (IM.IV)  (,-  (-.0),-)  T EH'  (,-  ( ,0).-)  SH  1115  N 

((,-  (-.0).-)  (0,0).DX)  (IH.IY)  VI  (AVL.O)  AVC  (AVR.O)  S iri6  {214), (2,0)  S)  CTH.C) 
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DIAGNOSIS 

DIALOGUE 

DICK 

DID 

DIDN'T 

DIWLNSKiNAL 

DIHCCrEO 

DISCUSS 

DISCUSSCI) 

DISCUSSES 

DISCUSSING 

DISPLAY 

DO 

DOES 
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DONE 

DOUG 

DRAGON 
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DREW 
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DURING 
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EARNEST 

ED 
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EIGHT 

eighteen 

EIGHTY 

El  ICfNONIC 
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El  I Vt  N 

El  I iOT 

EHGLIGH 

ENVIRONMIlU 

LRIK 

EWMAN 

ERNST 

EUGENE 

EVALUATION 

evaluator 

EVENTS 

EVER 

EVERY 
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«-  (-.0),-)  (D,0>,OX)  {AVL,0>  AVC  (AVR,0>  IH3  (>-  (-,0),-)  N 0W2  S 1H4  S (HH,0> 
((*■  (D.oi.nxi  (AVl.O)  AVC  (AVR.OJ  AA  Ei.3  AO  (►  (-,0),-)  (G,0) 

ll-  (-.0),-)  (0,0), DX)  1H2<  (-.0).-l4:)  (K.O) 

((►  (0,0), DX)  (IlIZLlHa*)  <(.-  (-.0),-)  (0,0),DX,0) 

«-  (-,0).-)  (0,0), OX)  IH3'  (4-  (-,0),-)  (0,0)  1H5  N ((-  (-,0),-!A))  (T,0),OX) 

((•-  <-.0),O  (0,0), OX)  1112  M EH2'  N (-,0),-,0)  SH|,10)  EN  EL3 

((-  (-.0).-)  (0,0), DX)  CR  R £H3‘  H-  (-.0),-)  T.DX)  (IH3,IX)  (-  (-,0),-)  (0,0H.0) 

((*-  (-,0),-)  (0,O),0X)  Ilin  S - {K.0)  AA2I  S (HH.O) 

((-  (-.0),-)  (0,0),DX)  1114  S - <K.0>  AA2'  S -14)  (T.O) 

((*•  (-.0).-)  (D,0),DX)  1H4  S - (K,0>  AA2'  S IHG  <Zi'4),(Z,0)  S> 

((-  (-,0),-)  (0,0), OX)  IH4  S - <K,0)  AA2I  S IMS  N'X 

((-  (-.0),-)  (n.O),OX>  (IH4.0>  S - (P  L.PL  (L,0))  (EVL,0)  EVC  (EVR,0) 

(-  (-.0).-)  D':21  (IH2,n)  LIX;6,I2) 

((-  (-,0),->  (D,0).OX)  AH2'  (214;,(Z,0)  S) 

((►  (-.0),-)  (0,0), OX)  AX'  (Zi4:,(2.0)  S)  AX  U ((.-  (-,0),-i<5))  (T,0),DX> 

((<-  (-,0),-)  (0,0), OX)  OW  U (EVL.O)  EVC  (EVR.O)  (N.DX) 

((►  (-,0),-)  (n,o),ox)  0W2  owi  N ((.-  <.,o>,-;4;)  (T,o),i)x> 

((.-  (-.0),-)  (D,0>,DX)  AA*  (N,DX)  EL2  (-  (-.0),-)  iD,0) 

«-  (-.0),-)  (r),o).ox)  AA2';,?2i  (n,ox) 

((-  (-.0),-)  (0.0), DX)  AA2'  (►  (-.0),-)  (G,0> 

(-  (-,0),-)  DR  H AC'  (.  (-.0).-)  (G.O)  IHS  N 

(-  (-,0),-)  08  R AE'  U (-.0),  ) (G.O)  1H3  N lZ;4;,(2,C)  S) 

U (-,a),-»  DR  R AA'  Ih'i  NX  (Zl4;,(2,0)  S> 

(4-  (-,C>.-)  DR  R flH.O)  UW> 

I-  (-,0),-)  DR  a (AVL.O)  AVC  (AVR.O)  f AH  S (HH,0> 

(-.0)  ({-  (-.0)..)  SH.fi)  (AVL.O)  AVC  (AVR.Oi  V (iH3,lV)  NX 
((.-  (-.0).-)  (D.O).DX)  ER  1V2'  NX 

((4-  (-,0),-)  (n,0),DX)  (AVL.O)  AVC  (AVfi.O)  N AES  M IH3  (4.  (-,0),.(4))  (K,0) 

(-.0)  IV  (.-  (-.0).-)  SH  (HH.O) 

(-.0)  ER2'  EL2 

(-.0)  ER'  I.  !V  (IX.O)  S .14)  (T,0) 

(-.0)  ER  (N,t)X)  1H6  S .;4)  (T.O) 

(-.0)  Eli?'  (-  <-,0).-)  (0,0) 

(-.0)  EH2I  ((►  (-.O),-)  (0,0), OX)  (11(5  N.EN>  (-  (-,0),-,Q)  (U.O)  EM2 
(-.0)  CH4  (-.0)  F 1112'  SH  IHS  N (<4.  (-.0),-)  (O.T>,OX)  L iV2 
(-.0)  (E  VL.O)  EVC  (EVR.O)  ((4.  (-.O.-lfl))  (T.OI.DX) 

(-.0)  (EVL.O)  EVC  (EVR.O)  ((-  (-.0),-)  T.DX-  IV  (N.DX) 

(-.0)  (EVL.O)  EVC  (EVR.O)  ((*■  (-.0).-)  T.DX)  IV 

( ,0)  (IV  I .El  3)  EII21  (-  (-.0).-)  (T.GH;,10J)  R AA?  N IH3  (-  (-.0>,-l4;)  (K.O) 

(-.0)  (!Y  I . Kl  :()  f H?'  (-.0).-)  (T.SHI.IO;-  R AA?  N 1h3  (-  (-,0),-)  S (HH,0) 

(-.0)  TH  (i,rn  EH'  V (ihs  n.en) 

(-.0)  EH  EL3  IV  iH4  ((-  (-,0),-;4;)  (T,0).DX> 

(-.0)  (1H3.IV)  NX  (.-  (..0).-,0)  (G.O)  L IHP'  SH  (HH.O) 

( ,0)  iH3  N V (AVL.O)  AVC  (AVR.O)  ER'  N M iHb'  (-  (-.0),-)  (T  O! 

'-,0)  (AE7.1  ll)  t H>  IIIH  (-  (-.0),-;4;)  (K.O; 

(-,C)  ER  M IH5'  (N.nX) 

(-.0)  1M8  ER'  N S -•/•]  (1.0) 

(-.0)  Y uw;>'  (^  (-.0'.-)  sii;  io;  'v  iN.dx) 

(-.0)  me  V AA  Eia  V ijW  (EVL.O)  EVC  t£VR,0)  SH  !HS  N 

(-.0)  me  V AA  ELS  Y UW  (EVL.O)  EVC  (EVR.O)  (I-  (-,0),-)  T.DX)  ER 

(-.0)  (lY.AX)  V EH4'  N (.-  (-.0).-)  S (HH.O) 

(-.0)  !H4i  V £K2 

(-.0)  (EH, Ell?)  V Ri  1V2 

(-.0)  (EH.KH2)  (V,0>  Ri  1Y2  TH  (IH3.IV)  NX 

(-.0)  EH  (4.  (-.0),-)  (Z;4:.tZ.C)  S)  AC  M (.-  (-,0).-,0)  P EL 

(-.0)  EH  <4-  (-.0),-)  (2;4:.(Z,0)  S)  AE  M (4-  (-.0),-,0)  P EL  (Z14}.(Z0)  S) 

(-.0)  ih3  (-  (-.0),-)  (z:4;.(Z.o)  S)  IH7I  s -m;  (T.o; 

(-.0  EH2  (.-  (-.0),-)  S'  - P ER  ((.-  (-,0).-;a;)  (T.o.dx) 


50 


OiC<iori«ry 


fXPLANATION  (- 

EXPRC55IONS  (- 

FABLES  (- 

FACES  (- 

FACTS  (- 

FAHIMAIM  (- 

FAIRY  (- 

FASTER  <- 

FEATLIPr-nHIVENE 


FEIIRUARY 

FEOKRAL 

FEICEMHADM 

FEl  OMAN 

FICTION 

FIFTEEN 

FIFTY 

FIKIS 

FILE 

FINISH 

FINISHEC 

FIRST 

FIVE 

FOR 

FORESTS 

FORMAL 

FORMATION 

FORTY 

FOUR 

FOURTEEN 

FRAME 

FRAMES 

FROM 

FU 

FUNCTION 

FUNCTIONS 

Flj;?ZY 


GAME 

(-.0),-) 

GAMES 

i- 

(-.0),-) 

CARY 

(- 

(-.0).-) 

GASCHNIG 

(- 

(-.0).-) 

GENERAL 

(-.0'.-) 

GENERATE 

(«-  (-.0),-) 

GENERATION 

(► 

(-.0),-) 

GEOMETRIC 

{<- 

(-.0).-) 

GEORGE 

(.- 

(-.0).-) 

Gf.V 

(.- 

(-.0),-) 

Gins 

(- 

(-.0).-' 

GIVE 

(- 

(-,01.-) 

GIVEN 

(- 

(-.0).-) 

CM 

{<- 

(-,0).-) 

GO 

(- 

(-.0),-) 

GO-MOkU 

(.- 

(-.0).-; 

GOAL 

(- 

(-.0).-) 

GOALS 

(.- 

(-,01.-) 

CRAIN 

(- 

(-.0),-) 

GRAMMARS 

(.- 

(-.01,-) 

GRAMMATICAL 

(.- 

(-,0),-) 

GSAr  n 

f /A  \ \ 

,0)  EH  (-,0).-)  S - <P  L.PE  (L.OM  IH  N (EVL.O)  EYC  (£VH,0)  SH  IHb  N 
,0)  IH3  <►  <-.0).-»  S - (P  R,PH  EH3'  SH  (IH5  N.EN)  (ZIAJ.IZO)  S) 

,0)  F (EYL.O)  EVC  (EVR.O)  (-.Ol.-IA})  (B,0)  AX  <L.O)  (Z{A;,(Z,0)  S) 

,0)  F (£Vi  ,0)  EYC  (EVR.O)  S IHA  (ZiA),(Z,0)  S) 

0)  F AC!  (-.0),-)  S <HH,0) 

,0)  F AO  EL2  M UHA  (N,OX) 

,0)  F EH'  (£«,H>  lY 
.0)  F AES'  S - T EH 

.0)  F lY  (-.0).-)  SH-.IO}  ER'  (-  (-.0).-)  OR  R IH  V («IM,|X)  N),£N) 

(-.0)  F EH3'  (.-  (B,0>  (R,V)  (UW  (W,0>,0)  AA  (E«,H;  lY 

(-.0)  F EH'  <-,0I.-)  O.OX)  £R2  EL3 

(-.0)  F (AYL.O)  AVC  (AVfi.O)  <-,0>,-)  (G,0)  IH5  N (-,0),-,0)  <B,0>  (AWL.O)  AW"'  (AWR.O)  M 
(-.0)  F £H2'  EL  (-.0> M IH6  (N.OX) 

(-.0)  F 1H2  (-  ( .0).-)  SH'  IHb  N 

<-,0)  F 11(2'  F <(-  (-.0).-)  T.OX)  lY  (N,OX) 

(-.0)  F 1H2I  F ((-  T.DX)  lY 

(-.0)  F (AYL.O)  AYC!  (AVR.O)  <-,0),-)  S (HH.O) 

(-.0)  F (AYL.O)  AYC  (AVR.O)  EL3 
(-.0)  F 1113'  N 1H5  SH  (HH.O) 

(-.0)  F 11(3'  N IH5  SH  -|4l  (T.O) 

(-.0)  F (AA3  ER,I  R2)  S'  -jA;  (T.O) 

(-,0)  F (AYL.O)  AYC  (AVR.O)  (V.F) 

(-.0)  F (AA4.0)  EH' 

(-.0)  F AA2  ER!  IH  S - S (HH.O) 

(-.0)  F AAA'  ER2  (M.(l)  ri2 
(-.0)  F AAA  ER  M (EYL.O)  EVC  (EYR.O)  SH  !H5  N 
(-.0)  F AU-.A'  (eR.EH2)  ((.-  (-.0).-)  T.DX)  lY 
(-.0)  F AAA'  ER 

(-.0)  F AAA'  ER  ((►  (-.0).-)  T.DX)  lY  (N,DX) 

(-.0)  F R (EYL.O)  EVC  (EYR.O)  M 

(-.0)  F R (EYL.O)  EYC  (EVR.O)  M (2|A;.i2,0>  S> 

(-.0)  F R AH'  M 
(-.0)  F UW2I 

(-.0)  F AA'  NX  (^  (-.0).-.0)  SH  1H5  N 

(-.0)  F A/O  NX  (4.  (..0).-.0)  SH  IHb  N (Zi4;,(2,0)  S) 

(-.0)  F UH2'  (2{4;-,(Z.O)  S)  lY 


Cb  SH  N IH3  (.-  (-.0).-14;)  (K,()) 

EH2  N CR2'  EL3 

EH2  N ER  (EYL.O)  EVC  (EYR.O)  ((-  (-.Ol.-JA;)  (T,0).OX) 
Ills  N EH  (I  VL.O)  EVC  (EYR.O)  SH  IHb  N 
IV  IH  M EM2'  (-  (-.0),-)  DS  R iHB  (►  {-,0),-(4;)  (K,0> 
UW4  ERl  (.-  (-.0),-)  SMI.IO! 

:H2'.1H3';  (I-  (-.0),-!4;.)  (T.O),DX) 

13'  (-.0).-)  S (HH.O) 


R (EYL.O)  EVC  (EVR.O)  (N.DX) 


\ /r*  n Ar'i*  r 
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GP/iPHiCS 

HAMHURG 

HAN'S 

HAPPEN 

HARRY 

HAS 

HASN'T 

HAVE 

HAVEN'T 

HAYES-ROTH 

HE 

HEARSAY 

HEl.l' 

HENDRIX 

HEH 

HERN 

HERPERT 

HEVERUSTATJC 

HEURISTIC 

HEY/ITT 

HILARY 

HILL 

HIS 

HISTORY 

HOLLAICO 

HOW 

HIJCfl 

HUMAN 

HUN'ORCO 

HUNGRY 

HUNT 

HYPOTHESIS 

I 

I'D 

I'M 

lEEC 

IFiP 

IJCAI 


(-.0),-)  (C,0)  R AE2'  E 1H2  S (HH.O) 

(-.0)  <HH,0)  AA'  M <e,0)  E«2  (-.0),-)  «'G,0) 

(-.0)  (HH.O)  AA'  N (-.0)  S (HH.O) 

(-.0)  (HH.O)  AC'  {-  (-.0).-)  P EH2  (N,OX) 

(-.0)  (HH2,HH>  AE2'  (EH  H)  IY2 
(-.0)  (HH.HH2.0)  AE4'  CZ'.A'JZO)  S) 

(-,0)  (HH.HI12,0>  AE'  (Z;4:.(7.0)  S)  IH6  N ((^  (-.0),-(4})  (T,0),DX) 

(-,0)  (HH.HIt2,0)  (AE'.AE'j'i  V (F.O) 

(.,0)  (HH,H1I2,0)  A£I  <V,4-)  (EH2  N.EN)  ((^  (-.0),-|4;)  (T,0),DX) 

(-.0)  (HH.HH2.0>  (EYL.O)  EVC  (£VR,0)  (Z{4;,(2.0)  S)  R AA  TH  (HH,0) 

(-.0)  (HH,HH2,0)  lY' 

(-.0)  <HH.HH2.0)  1V2  ER  (-.0)  S (E'VL.O)  EVC  (EVR.O) 

(-.0)  (HH.O)  AA3'  EL3  (-  (-.0).-)  (P.O) 

;-,0)  (HH.O)  EH2  N ((^  (-.0).-.0)  (OR.O).OX)  ER  11(2'  (-  (-.0),-)  S (HH.O) 

(-.0)  (HH',HII2i)  EH' 

(-.0)  (HH.O)  iW  (.-  (-,0).-14:)  (B.O) 

(-.0)  (HH.O)  EH2'  (.-  (-.0).-(4:)  (B.O)  EH  U*-  (-.OI.-jA')  (T,0),DX) 

(-.0)  (HH,0>  EH  (-.0).-)  DR  R OW  S - T AC  ((►  (-.0),-)  T.DX)  IH3  (*-  (-,0),->  (K,0) 
(-.0)  (HH.HI(2.0)  (FH3.)Y?.0)  ER  IH2'  S (-  T.O)  IH2  (^  (-,0),-14})  (K,0> 

(-.0)  (HH.HI12.0)  V UW  !h4  (-  (-.0>.r)  (T.O) 

(-.0)  (HH,HI(2.0)  ELS  ER'  IV2 
(-.0)  (HH.Hll2,0)  AH3'  LL4 
(-.0)  (HH.O)  IH3'  (Z:4;.(7,0)  S) 

(-.0)  (HH.O)  !H3‘  S - OR  ER  IV 

(-.0)  (HH.O)  AA'  tl  3 EN  (4-  (-.0).-.0)  <D.O) 

(-.0)  (HH.III)2.0>  (AWl.O)  AWC  (AWR.O) 

(-.0)  (HH.HI.'2.0>  IH2  UH' 

(-.0)  (HH.HH2,0>  UW3'  Ml  liH  (N.OX) 

(-.0)  (H)l,0)  (AA0'.AA2')  N (.-  (-.0).-)  (OR, 0.0)  ER  (-.0).-)  (0.0) 

(-.0)  (HH.O)  UH2I  NX  (.-  (-.0),-)  (G.O)  R IV3 
(-.0)  (HH,0)  UH2'  N ((.-  (..0).-(4;)  (T.OI.DX) 

(-,0)  (HH.O)  (AVL.O)  AVC  (AVfl.O)  (.-  (-.0).-)  P AA  TH  IH6  S IH6  S (HH,0) 

(-.0)  (AYL.O)  AVC  (AYR.O) 

(-.0)  (AVl.O)  AVC  (AYR.O)  (i-  (-.0).-)  (0,0) 

(-.0)  (AVL.O)  AYC  (AYS.AYX.O)  M 

(-.0)  (AYL.O)  AVC  (AYR.O)  ((-  (-.0).-)  T.OX)  R l.H'  (.-  (-.0),-)  P (EL/(X  EL)  IV 
(-.0)  (AYL.O)  AVC  (AYR.O  F IH2  (.-  (-.0),-)  (P.O) 


(-.0)  (AVl.O)  AYC  (AYR.O)  (-  (-.0).-)  SHI.IO)  (EVl.O)  EVC  (EVR.O)  S IV  (EVl.O)  EVC  (EYR.O)  (AVL.O)  AVC  (AYR.O) 
EL3  UK4  N (OVL.O)  OVC  (OVR.O) 

M'  !H3  (.-  (-.0).-)  SHi.s; 

M'  IH3  (-  (-.0),-)  SHi.B)  IH3  (Z;4:,(Z,0;  S) 

M (4-  (-.0)  -.0)  P'  R (IH,0)  UW'  V (IH3.IY)  NX 
.IHC.IX')  (N.OX) 

N (-  (-.Ci),-,0)  (0,0)  AA3  (-  (-.0),-)  T IH2  V (F.O) 

N (-,0).-.0)  (0.0)  AA2'  5 - OH  (H.O)  ii2  EL3 
N IH31  (-  (-.0).-)  (Z;4;,(Z,0)  S)  At  ((-  (-,o),-!4;)  (t.o.ok) 

N F'  R ;t(6  N S (HH.O) 


ILLINOIS 

(-.0) 

IMAGE 

(-.0) 

IMAGES 

(-.0) 

IMPROVING 

(-.0) 

IN 

(-.0) 

INDUCTIVE 

(-.0) 

INDUSTRIAL 

(-.0) 

INEXACT 

(-.0) 

INFFWI  Ncr 

(-.01 

INFFHI  l.'CES 

(-.01 

IN)  EHENriAl 

(-.0) 

INFORMATION 

(-.0) 

INhEH)  lAN'CC 

(-.0) 

INSAIVE 

(-.0) 

INSTITUTE 

(-.0) 

INTEUIGENCE 

(-.0) 

INTEll  IGfNT 

(-.0) 

INTENSITY 

(-.0) 

INTENTIONS 

(-.01 

T IH4  (-.0).-)  T LiWS'  ((-  (-,0),-'.''<;)  (T.O).DX) 

(-.0).-)  T.DX)  EH  El  IHC  (►  (-.0),-)  SH  1H3  N S (HH.O) 

(-.0),-)  T.DX)  EH  EL  IHB'  (►  (-.0).-)  SH  iH3  N [<-  (-.0),-)  (O.T) 
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OlCliC'r%»ry 


INTtWACTlVE 

INTtKESTtO 

INTEHPHCTABIE 

INTtKPBETIVE 

INTEMHUPTS 

IMEPVICW 

INTOKATION 

INVARIANCE 

INVARIANCES 

INVESTMENT 

INVOCATION 

IRV 

IS 

ISN'T 

ISOMEHS 

ISSAC 

ISSUE 

ISSUED 

ISSUES 

IT 

ITERATION 

ITS 

JACi< 

JAMES 

JANUARY 

JEAN 

Jt  ( I REV 

J(  REV 

JOHN 

JOStl'U 

JOURNAL 

.'OURNALS 

JUDEA 

judicial 

JULY 

JUNE 

KARL 

KEITH 

KEN 

KEY 

KEYS 

KILL 

KINO 

KINDS 

KING 

KNC'..' 

KNOWLEDGE 

KNOWN 

KUGEL 

LASS 

LAMmDA 

LANGUAGE 

LANGUAGES 

LARGE 

LAST 

LATELY 

LATEST 

LAURENT 


i-,0)  IH3  N ((.-  (-.0),-)  DH.DX)  £R  AC3  (.-  (-.0).-)  T iH3  V (E.O) 

(-,0)  IH3'  N «►  OR,nX)  £R  S <-  T.O)  lrl3  {-  (-.01,-)  (0,0H,0) 

(-.0)  IH7  N <-.01.-5  OR.OX)  ER  (*-  (-.0),-)  (P,0)  Ek2i  ((.-  (-.0),-)  T.OXl  UW«  (.  . ..0),-{4))  <B,0)  EL 

(-.0)  IHS  N <-,05,-5  DR.OX)  ERO-  (-,0),-,0)  P ER  <(^  <-,05,-5  T.DX)  IH2  V <F.O) 

<-,0)  IH3  N ((o  <-.0).-.Ci  DR.OX)  £R3i  AA2  <,-  (-,0),-)  3 <HH,0) 

<-.0)  IH5  N <-.0),-)  DR.OX)  ER'  (V.F)  Y UW 

<-,0)  IH5  N (-  <-.0).-)  T UH  N'  (£YI,0)  £VC  <EVR,0)  SH  IH5  N 

<-.0)  1H3  N V EH3  ER'  lY  <IHA13).0>  N <-,0>  S (HH.O) 

<-.05  IH3  N V £H3  ER'  lY  <IHA(3).0>  N (-.0)  S IH4  <Z{4},(Z.O)  S) 

<-.0)  iHb  N V EMA  S'  - M IHS  N (.-  (-,0>,-10}l  (T  0) 

<-.0)  IH3  N <-,0)  V{J)  iHB  <.-  <-,0),-)  <K,0>  <EVL,0)  EVC  (EYR.O)  SH  IHb  N 
<-,0)  ER2'  V (F,<i> 

<-,0)  IH3'  (214;.(Z.05  S) 

<-.0)  IH3'  (Z'AMZ.O)  S)  <1HC  N.EN)  (.,o>,-(4’,)  (T,0),OX) 

<-,0)  <AVl.O)  AYC  <AVR,0)  S EH4  M'  ER  <214;.<2.0)  S) 

<-.05  <AYl  ,0)  AYC  (AYH.O)  <2:4J,<2,0i  S)  !H3  <-,0),-!4})  (K.O) 

<-,0)  IH3  SH>  U\Y3;5! 

<-,0)  IH3  SH'  UWSIb)  <•-  <-.0>.-)  <0,05 
<-,0)  IH3  SH'  lAYSIb)  i7.'AH7,0)  S) 

( ,0)  IN')'  (-.0).-)  <T.O).  0X5 

<-,0)  ■i:3  u-  (-.05,-5  T.O'X)  ER  (tVL.05  EVC  (EYR.O)  SH  IHb  N 
(-.0)  IH4I  <-  (-,0).-5  S <HH.O) 

(►  (-.0),-)  SHI.  10!  AES'  (-  (-.0).-14;)  (K.O) 

(-.0).-)  SH;,10!  (EVL.05  EYC  (CYR.O)  M I-.O)  (Z:4;,(2,0>  S) 

(-  (-.0),-)  sit:, 10!  AtO'  N Y <CH.1H2)  (AA4,0)  (r.R.R)  |Y 

(-  (-.0),-)  sh;,io;  iv>  (n.ox) 

(-  (-.0),-)  sh;.io;  ehs'  e r iv2 

(<•  (-  .O),-)  SH[,10)  EH'  EH  IV 

it-  (-.0).-)  SHI.lOi  AA2'  (N.DX) 

it-  (-.0),-)  SHl.tO!  OW'  S IH  F (HH.O) 

ft-  (-.0),-)  SH|,50!  ER'  N (AA,0)  EL3 

<►  (-,0),-5  SHi.lO!  ER'  W (AA.O)  EL3  <Z;4;,(Z,0)  S) 

<^  (-.0),-)  SH!  101  LAYS'  (*•  (-,05.-)  (0,0)  lY  1H2 
if  (-.0),-)  SH;,10!  UW3' t-  (-.C.'-l  (0,0)  IHS  SH  els 
(►  (-.0),.)  SHI.IO)  AX'  : (AVI, 05  iVC  (AVR.O) 
if  (-.0),-)  SH'.IO)  (IH20)  UW  (N.0X) 
if  (-,0',-)  (K.O)  AA3'  EH2  E13 
if  (-.01.-)  (K,0>  lY'  TH  (HH.O) 
if  (-.0).-)  (K.O)  EH2'  (N.OXl 
if  (-.0).-)  (K.O)  iV' 
if  (-.0).-)  (K.O)  lY'  S (HH.O) 

(,-  (-.0),  ) (K,0)  IKS'  El 

(v  (-,(1),-)  (K,0)  (AVl.O)  AVC  (AVR.O)  .N  (^  (-.0),-)  (D.O) 
if  ( .0),-)  (K,<)>  (AVl.O)  AVC  (AVR.O)  N'  lO.D)  (2;4;,(2,0)  S) 
if  (-.0)  -)  K'  (IM3.:Y)  .NX 
(-,0)  H OW' 

(-.0)  N AA'  El  3 Illfi  if  (-.0).-)  SH;,121 
(-.0)  N OW'  (N.OXl 

if  (-.0),-)  (K.O)  uwE'  (-.0).-)  (G.o;  Cl 
(-.0)  L Af ' (-,o).-i4;)  (fl,n>  (z:a;,(2,0)  si 

(-.0)  L AE  M (,-  (-.0),-)  (0.0)  (AiEUH) 

(-.0)  L (A^A.AC4)  NX  i.-  (-.CD.-.O)  WH'  1H3  if  (-.0),-)  (2H  (SHi,  101,0), SH;, 10)) 

(-.0)  L (AA,AC4)  NX  if  (-.0),-,0)  V/H'  IH3  if  (-.0).-)  (ZH  (SHl,  101,0), SH!, 10))  lH?'!6,16i  (2|4;,(Z,0)  S) 
(-.0)  L AA  R (,-  (-,0).-)  SH  (HH.O) 

(-.0)  L At ' S -[’t]  (T.O) 

(-.0)  (l,(?)  (EVL.'-)  EVC  (EVR.O)  (^  (-.0),-)  (T  (-.0,0)  L lY 

(-.0)  L (EVL.C)  EVC'  (EVR.O)  Hf  (-.0).-;4;)  (T.O.'.DX)  IriO  S -;4)  (T,0) 

(-.0)  L AAA  ERZ'  Ih4  N (^  (-.0).-)  (T,0) 


53 


Dicliori^y 


ICARNING 

ucTuRrn 

Lfi: 

Lf.fMT 

LtOMARO 

LCS 

LrSStH 


<-,0)  L2  ERi  N IH7  MX 

< .0)  L EH  (-.0).-)  SHil.lOl  ER  (Z:a;,(Z,0)  S> 
(-.0)  L IV' 

<-,0>  L AAZi  (N,OX)  AH2  ((►  (-.O.-lA’l  (T,0),CK) 
( ■,0)  L AA2  EM'  EH  ((►  (-.0),-)  (0,0), 0X> 

<-,0)  L Utiai  G <HM,0) 

(-.0)  L AH'  S Efi2 


LEV 

LET'S 

LCXICOMETHY 

LIGHT 

LIKE 

LIMIT 

LIMITED 

LIN'DA 

LINE 

LINEAR 

LINGUISTICS 

LISP 

LIST 

LISTED 

LISTING 

LOCATION 

LOCATIONS 

LOGIC 

logical 

I ONG 

LOSING 

lOW 

MACfllNI 

MACEINES 

MACRO 

MAOtl  INE 

MAGAZINES 

MAUL 

MANAGEMENT 

MANIPULATING 

MANIPULATORS 

MANNA 

MANTRA 

MANY 

mapping 

MARCH 
MARKET 
MARR 
MARSlANO 
MARTELLI 
MARVIN 
MARV 
MASINTEH 
MASSACHUSETTS  (• 
MATCHING 
MAV 

MCCARTHY 
MCCOHOLXIK 
MCDERMOTT 
ME 

MEANING 


(-,0)  L AH?'  {{.-  ( .o.-i'a;)  a.o),ox) 

(-,0)  L AH2'  (-.0),-)  S (HH,0> 

(-,0)  L EH  (-  (-.0).-)  S IH2I  («.  (-,0),-)  (K.0)  AA  M UHfl  (-  (-,0).-)  OH  R 1Y2 
(-,0)  L (avl.o)  ayc!  (avr,o>  a*,  (-.o),-!*;-)  (T.o),dx> 

(-,0)  L (AVL.O)  AYC  (AVR,0)  (►  (-,0),-{4;)  (K,0) 

(-.0)  L IH  M UHAi  {(►  (-.0).-:4;)  (T,0),DX) 

(-,0)  L IH  M UHRi  «-  <-,0),-)  (T.0,0),  DX>  IhA  (►  (-.0),-)  (0,0) 

(-.0)  L IHG  N‘  (.-  (-,0),-.D)  (0,0)  UH4 
(-,0)  L (AVL,0)  AYC*  (AVfl.O)  (N.OX) 

(-,0)  L iH3'  N IV  E!» 

(-,0)  L 1H3  NX  (►  (-,0).-;  '-VH  IH  S - T IH3  V (-,0),-)  S (HH,0> 

(-,0)  L IHG'  S - (P,(l) 

(-.0)  J.  IHG'  (Z[4;,(Z,0)  S)  (-  (T,0),C) 

(-,0)  L 1H6I  s - T (IX,IH4)  (.-  (-.0).-)  (O,OH,0) 

(-,0)  L UHA  S - T'  (IH3.IY)  MX 

(-.0)  L OWS  (.-  (-.0).-)  (X,())  (EVL.O)  EVC  (£V#,0)  SH  Hib  N 

(-,0)  L 0W3  (-  (-.0),-)  (K.O)  (fVL.O)  tVC  (EV6,0)  SH  lllb  N (Z|4),(2,0)  S) 

(-,0)  L AO  (-,0),-)  sh';,h;  ih3  (-  (-.o).-;4;)  (k,o) 

(-,0)  L AO  (.-  (-,0)..)  SHi;,H;  |H3  (.-  (-.O).-)  (K,0>  el 

(-,0)  L2  0W4'  NX 

(-,0)  L UW^  (2!4;.(Z,0)  S>  (IH3,1V)  NX 
(-,0)  I ow 

(-.0)  W IHb  SH'  iV  (N,I)X) 

(.,01  M IHS  SH'  IV  N (Z'4;,(Z,0)  S) 

(-,0)  M A( (^  (-.01,-)  (K.O)  R OW 

(-,0)  M At ' «4-  (-,0),-)  (0,0;,0X)  AH  El  IHC  (ROX) 

(.,0)  M A(',i  (.-  (-,0),-)  (G,0)  1H4  (Zi4;,(Z.O)  Si  iV  N (Zi'T’.iZ.O)  S) 

(-.0)  M (EVI.O)  EVC  (EVR.O)  (<-  (-.0>,-1'T;)  (K,0> 

(-,0;  M Af'.i  (DX,N)  EH2'  SH;,8)  M EH2  N (<►  (-,0>,-;4;)  (T,0),DX) 

(-.0)  M IHb  N 1H3  (.-  (-  .O),-)  P Y UH2  L (EVL,0>  EYC  (EYR.O)  ((-  (-,0),-)  T,DX)  (1H3,IV)  NX 

(-.0)  M IH'j  N IH3  (.-  (-.0),-)  P V UW2  L (CVL.O)  EYC  (EYR.O)  ((-  (-,0),-)  T,DX)  ER  (Zi4;.(Z,0; 

<-,0)  M AA'  N AA 

(-,0)  M AO  IJi  (-.0),-)  DR  R iH2 

(-,0)  M (th2,i:H3>  (N,OX)  lY' 

(-,0)  M ALS  {.-  (-.0).-)  P (lH3.iY)  NX 
(-.0)  M AA'  H (.-  (-,0),-)  SH  (HH.O) 

(-,0)  M AA2  ER2'  (^  (-,0',-)  (K.O)  IH3  ((►  (.,0),-;a;)  (T,0>,DX'> 

(-,01  M AA'  ER2 

<-,oi  M AO  ER  (Z;4;,(Z,0)  si  l uho'  n <-,oi,-,oi  (O.oi 
(-,01  M EH2'  (»  (-,0).-)  T ELS  iV2 
(-,01  M AA  Ri  V IH  (N,I)X) 

(-,01  M AE/'  EH  1Y2 

(-,01  M UHfl  S EH2'  N (-,0).-t  T,OX)  ER 
01  M AES  S IHb  (.-  (-.01,-)  SH'.B!  UWS'  S iHfl  (^  (-.Cl,-)  S (HH.O) 

(-,0)  M AE'.i'  (-  (-.0).-)  SHI.IO)  IH'j  NX 
(-,0)  M (EVL.O)  EVC  EVR* 

(-.0)  M AAS  (.-  (-.0).-)  iK.O)  AA3'  E»2  TH  IV 

(.,0)  M (-  (-,0),-)  (K.O)  AAfl'  EH  iw  ■-.0}.-,  D.OXl  IHB  (►  (-,0),-i4i)  (K.O) 

(-.0)  M 1H3  (►  (-.0).-)  OH  EH  M EH4  (-,0).-)  (T.Ol 
(-,0)  M IVi 

(-,0)  M IV  N'  (IH3.1V)  NX 


S) 
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Dicfionvy 


MKAkS  (-.0) 

MKIUCAl  (-.0) 

Mill  INC  (-,0) 

MftVlN(;S  (-.0) 

MKilXER  (-.0) 

MEMORIES  (-.0) 

MEMORY  (-.0) 

MENTION  (-.0) 

MENTIONED  (-.0) 

MENTIONING  (-.0) 

MENTIONS  (-.0) 

MENU  (-.0) 

MENl.1?  (-.0) 

MEVA.SVMROLIC  (-.0) 
MEIAMATHEETATICS 


MEIHQDS 

MICE)  ACL 

M1CHAL5KI 

MICHIC 

MIKE 

MINIMAL 

MINKEK 

minse;y 

Ml  TC  HE  Li. 

Ml  ISP 
Ml  ISP? 

MODEL 

MODEtlNG 

MODEtS 

MONITOR 

MONKEV 

MONTH 

MOrjTEIS 

MORE 

MOST 

MOSTOW 

MOTION 

MOVE 

MOVEMENTS 

MOVIES 

MDLTILCvEi 

MDI.TIPKOCLSS 

MUSIC 

MUST 

MVSELi 

NACEl 

NASli  WLiTBER 
NATION/il 


(-.0) 
(-.0) 
(-,0) 
(-.0) 
(-.0) 
<-,0) 
(-.0) 
0.0) 
(-.0) 
<-,0) 
(-.0) 
(-.0) 
(-.0) 
(-.0) 
(-.0) 
<-.0) 
(-.0) 
(-.0) 
(-.0) 
(-.0) 
(-,0) 
(-.0) 
(-.0) 
(-.0) 
(-.0) 
(-.0) 
<-,0) 
(-,0) 
(-.0) 
(-.0) 
(-.0) 
( ,0) 
( 


M IV  N*  (2;4:,(?.0)  S) 

M EH  <(^  (-.0).-)  (0  0), OK)  III?  w <-,0),-14;)  (K,0)  EL 
M ivt  ((^  <-,0).-)  T.OX)  IH7  NX 
M lYi  ((-  (-.0).-)  T.DX)  IH7  NX  (Z,  1,S)) 

M AAZ  EL3'  (.  (-.0).-)  S ER 
M EH2I  M (OW4.0)  (fi.EH?)  iYZ  (Z(S,0>.S) 

M EH2'  M (0W4.0)  («.E«2>  IYZ 

M (EH2.1H)  N'  {t-  (-.0).-.0>  SHI,  101  IH5  N 
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.0)  R IV'  (2(a;.(Z.o)  S)  ckc.o)  n (iH3,iV)  nx 
,0)  R IV2'  S (Ills  N,EN)  ((-  (-.0),.;4:)  (T.OI.DX) 

,0)  R r/2'  S (IHG  N.tN)  «-  (-.O).-;^!)  (r,0),DX)  L IV 
,0)  R EH3'  i-  (-.0).-)  (K,0>  IX  (►  (-.0).-)  N IH3  SH  IH5  N 
,0)  R;.I5I  EH2'  ((►  (-.0).-)  (O.OI.DX)  IV 
,0)  R 1V2  ((.  (-.0).-)  D.OX)  AAZ'  (-.0).-)  SH  IH5  N 
.C'  S IV  (^  (-.0).-)  uQ.O; 

,0)  (R.O'  iVZ'  (-.0)  I tfi 

,0)  H EH3'  (-.0)  F ER  N S (Hh.O) 

.0)  R EH3'  (.,0)  FERNS  -|A;  (T.O) 

.0)  H EH3I  (-.0)  F ER  N S IHC  <2,V;;,(Z.O)  S) 

.0)  fi  IV2'  .-.0)  f Eft  (4.  <-,0),-)  (0,0) 

.0)  R IV2'  ( ,0)  E ER  (Iri3.1V.O)  NX 

,0)  ft  1V2'  (-  (-.0).-)  (G.O)  AA2  ER  (DX,0)  IH5  NX 

.0)  R IV  (-  (-.0).-)  SH{.8i  IKb  N 

.0)  fi  EH'  (-  (-,0).-)  (G.O)  (Y.O)  UW  (EL.l)  ER  I lY 

.0)  R IH2  ((►  (-.0),-)  T.DX)  ER 

.0)  R (IH3.IV2)  I (EVL.O)  EVC  EVfi'  ((►  (-.O),-!-};)  (T,0),DX) 

.0)  H <!HaiY2)  t (EVL.O)  EVC  EVR'  ((-  (-.O),-)  T.DX)  (IH3,IX)  (►  (-.0).-)  (0.0H,0) 

.0)  R (IH3,IY2)  I (EVL.O)  EVC  EVR'  (-  (-.0),-)  S (HH.O) 

.0)  R lY  L (EVL.O)  EVC  (EVR.O)  SH  N EL 
,0)  R (1H3.IV2)  I IV  S -!A;  (7.0) 

.0)  R 1113  (-  (-.0).-)  P AAA'  ER  ((-  (-.O/.-.'A;)  (T,0),DX> 

.0)  R 1113  <-  (-.0).-)  P AAA-  Eft  ((-  (-.0),->  T DX>  ER 

.01  R (IH3.IY?)  (.-  (..0),.)  P OW'  ER2  <(-  (-,0),-)  T.OX)  ER  (Z[AJ,(Z,0)  S) 

,0)  R (1M3.IV7)  <►  (-.0).-)  P OW'  («.ER2>  (*-  (-.0),-)  S (HH.O) 

,0)  R EH3  (.-  (-.0),-)  (P  R.PR  (H.(l»  1H2I  (Z1A:,(Z,0)  S>  IHA  N (-  (-,0),-)  T (EVL.O)  EVC  (EVR.O)  SH  IH5  N 

,0)  R EH3  (.-  (-.0).-)  (P  R.PR  (H.O)j  IVI2'  (ZIAMZ.O)  S)  EHZ  N ((.-  (-,0),-)  T.OX)  (IH3.IV)  NX 

.0)  R (IH3,1V?.)  (►  (-.0).-)  (K,0)  OW  AE  S -lA)  (T.O) 

,0)  R IV  S EH'  (<-  (-.0).-)  SHl.lO; 

,0)  R IH6'  (Z;A;,(Z.0)  S>  UH7  <t(,U  0W2  SH  IH5  N 
,0)  R IV  S UWA'  ER  S (HH.O) 

.0)  H (ll(3.IV7)  S - (P.O)  AA'  N S (-.iX)  (ZIAMZ.O)  S) 

,0)  R (1H3,1V?)  S (-.0)  (OR  (ft.OJ.T  R)  IHZ-  ((-  (-.OL-IAJ)  (T.Ol.OX) 

.0)  (R  (1M3,IV2).EH.3)  (-  (-.O.'.-.O)  (Shi.  10), OH)  (R.O)  IY2'  V UW  EL 
,0)  R (lH3.iV2)  (-  (..0).-)  (SH'.lOj.T)  R IV  V lE.O) 

,0)  R (1H3.IV2)  (-.V)  (T.O)  IV  UW  (ZiA),(Z,0)  S) 

,0)  R AA  M (-  (-,0).-;A:)  (B.O)  Eft2  (►  (-.0),-)  (G.O) 

0)  R IH3’  (-  (-.0),-)  sh;,ioi 

.0)  R IH2I  (^  '-.0),-)  SHl.lO)  ER  (►  (-.0),-)  (0,0) 
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SCENE 
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SEARCH 
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SEEK 
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SENTENCE 

SENTENCES 
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SESSION 

SESSIONS 

SEVEN 

seventeen 

SEVENTY 
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SHAPE 

SHAW 
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SHOOTING 
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SHOW 
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(-,0)  R IIP  (K.O) 

(-.0)  S IY2>  (-  !-,0).-.0>  <G.O)  ER 

(-.0)  R IV  s - (8.0)  tHfl  (4.  <-.o>,-;a;>  (k.o> 

(-.0)  R (AVl.O)  AYC  (AYR.O)  (2141,(2.0)  S)  M EH2  (N.OX) 

(-.0)  R AO'  (4-  (-,C).-i4;)  (8.0)  ER  a-  (-.0),-;4})  (T,0),OX> 

(-.0)  R OW  (»-  (-.0).-|4;)  (8.0)  AO'  ((.-  (-,0).-{4;>  (T,0),DX> 

(-.0)  R OW  (-  (-,0).-(4:)  (B.O)  AO'  ((►  f-.C),-)  T.OX)  1H3  (-  (-.0),.(4:'  (K,0) 

(-.0)  R OW  (-  (.,0),-:fl;)  (B.O)  AO'  ((-  (-.0),-)  T,DX)  IH3  (-  (-.0).-)  S (HH.O) 

(-.0)  R OW  (»-  (-,0).-14;)  (B.O)  AA  (►  (-,0),-'  S (HH.O) 

(-.0)  R AA'  (.-  (-.0).-)  Sh.MOJ  EH4  S - DR  ER 
(-.0)  fl  AO'  (-  (-.0).-)  Sh;.»01  ER 
(-.0)  R AA'  (N.DX) 
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(-.0)  R AA  M EL?  (Hll.lH  AA3  ER  (-  (-,0).-)  (T.O) 

(-.0)  R AA5  (.-  (-.0).-)  (G.O)  £«'  (2;4;.i2.0)  S) 

(-.0)  R I'l  (-  <-.0),-)  N ER 

(-.0)  EA?  S AlC'  EL'  (-  (-.0).-)  (G,0)  R AE  F S (HH.C) 

(-.0)  S At  S ES  •(-  (-.0).-)  (0.0), OX)  AO  ((-  (-.0),-)  T.OX)  lY 
(-.0)  S AL'i  M Etl'1  (-.0),-)  (T.O) 

(-.0)  S AC4'  N (-  (-.0).-)  (0,0)  W EL? 

(-.0)  S AC  (T.OX)  IX  S E AC  («-  (-.0),-)  SH  IKS  N 
(-.0)  S (EVl.O)  tVC*  (£VR,0) 

(-.0)  S IV'  iN.OX) 

(-.0)  SH  ACS'  (-  (-.0).-.0>  (K.O; 

(-.0)  S (AYl.O)  AVC  (AYR.O)  !H5  N S (HH.O) 

(-.0)  S - (K.O)  AA3*  ((►  <-.0).-;4:.)  <T.O>,OX) 

(-.0)  S ER  (.-  (-.0),-)  5H  (HH.O) 

{.,0;  S IV' 

(-.0)  S (lY'.iX')  (-  (-,0;,-‘4:i  (K.O) 

(-.0)  S (lY.IV')  (,.  (-.0;.-)  (K.0)  (IH3,1Y)  NX 

<-,0)  S £H2  (.-  (-.0),-)  (G.O)  M UHA'  N (4-  (-.0),-)  T (EVL.O)  CVC  (EVR.O)  SH  IH5  N 
(-.0)  S (AX, IX)  L EH'  ;(►  <-,0)..|41)  (T,0),DX> 

(-.0)  S EH  £l2'  (-  (-,0).-i  S t« 

(-.0)  S IHG'  M ACS  N ((-  (-.0),-)  T,OX)  IH3  (-  !-,0),-:4;)  (K,0> 

(-,0)  S IhC'  M ACS  N :<►  (-.0),-)  T.OX)  lh3  (-  L-,0),-)  S (HH.O) 

(-.0)  S Eh  N S (HH.O 

(-.0)  S (tH2  (-  (-.0),-)  Elv.tEri?  N,EN)  (►  f-.0>.-.0)  T Eh?  N)  S'  (HH.O) 

(-.0)  S (EH?  (4-  (-.0).-)  EN.(EH2  N.EN)  (-  (-.0),-.0)  T EH2  N)  S'  IH6  (2(4;, (2,0)  S> 

(-.0)  S EH'  W (-.0).-)  T EH  M (.-  (-,0),-!4;)  (B.O)  ER 
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(-.0)  S (AhX.EH)  (-.0)  SH'  IHb  N 

(-.0)  S (AH?, Eh)  (-.0)  SH'  ihS  N (2;4;.(2,0)  S) 

(-.0  S AH?  V una  (N.IjX) 

(-.0)  S (AM?,EH)  V ((IhS, EH?)  N.EN)  <(*-  (-.O),-)  T.OX)  iV  (N.OX) 

(-.0)  S (AH2,E*I'  V'  ((inS.E'l?)  N.EN)  ((»-  (-.0),-)  T.DX.O)  IV 
(-.0)  S Eh?  V R El 
(-.0)  S IV  M'  OW  £k3 

(-.0)  SH  (tVL.O)  EYC  (EVR.O  (4.  (-.0).-)  (P.O) 

(-.O'  SH  AO' 

(-.0)  SH  IV' 

(-.0)  SH  iAV3'  (iT-  (-.0)0;  T.OX(  ■iiM.iY)  NX 
(-.0;  SI'  'V(4  £R2  i-  ( ,0'.-)  (T.O)  L I'l?  E (mm.O) 

(-.0)  SH  (i:l3.0)  iin4'  (■,„  (-.0/,-)  ((),0).0X) 

(-.0  SH  AA5'  (OW.O) 

(-.0)  S IH3'  (►  (-,0,-)  (GO)  AA3  (R.f  «:■  ((-  (-.0),-;4|)  (T,0),DX) 
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.0)  <5  !H3  (-.0),  ) iK.O)  I Ak7‘  5 IV 

,0)  S (AV[,0)  aYC  (AVR.O)  M CJMJ.IM-J)  (N.Dx, 
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SINCE  (-.0)  S (1H3,IH6)  K'  S \HH.O> 

SIX  (-.0)  S !H3i  (.-  (-.0).-)  S (HH.O) 

SIXTEEN  (-.0)  S IH'  <-,0).-)  S - T IV  (N.OX) 

SIXTY  (-.0)  S IH' (►  ( ,0).-)  S - T IV 

SIZE  (-.0)  S'  (AVL.O)  AVC  (AVR.O)  <2{4!.(2,0)  S) 

SLAGLE  (-.0)  S L (EVUO)  EVC  (EVR.O)  (-.0),-)  (G.O)  El 

StOW  (-.0;  S L OW 

SMC  (-.0)  E)M  S EH2I  M S IV 

SMITH  (-.0)  S M iti.  ;h  (HH,0) 

SHARING  (-.0)  S N (EYL.O)  EVC  (EVR.O)  EH  (IH3.IV)  NX 

SO  (-.0)  S 0W3' 

S08EL  (-,0»  S 0\v';i'  (►  (..3),.|ii;)  (fi.O)  EL 

SOFTWARE  (-.01  S AOi  F . 0)  (T.O)  V/  EB3 

SOLOWAV  (-.0)  S AO  EL  1 LIW2  W (EVL  0>  EVC'  (EVR.O) 

SOLUTIONS  (-.0)  S OW  L'  UW  SH  IHb  N (Z(4*.(Z.O)  S) 

SOL'VING  (-.0)  S AA' EL2  V IH5  NX 

SOME  (-.0)  S AA>  M 

SOMKIHING  (-.0)  S AAi  M TH  (li)3.IV)  NX 

SOMEWHEHi;  (-.0)  S AA'  M W Eh3  EH 

SORT  (-.0)  S UW4I  EH  ([-  (-.OI.-;'I;)  (T.O).OX) 

SORTS  (-.0)  S UWA'  ES  (^  (-.0),-)  S (HILO) 

SOURCES  (-.0)  S UWA  Efi  S 1H4'  (2:4:.(?.0)  S) 

SPACf  (-.0)  S - (M.O)  (EVl.O)  EVC  (EVR.O)  S (HH.O) 

spanning  (-.0)  S - (P.0,'  AES  (N.I)X)  (il(3.1V)  NX 

SPEECH  (-.0)  S - (P,0)  IV  (.-  (-.0)  -)  Sri  (HKO) 


SPEED 

SPHOULL 

SRI 

STANfOHO 

STATE 

STEHEO 

STEVE 

STOCHASTIC 

STOCK 

STOP 

STORAGE 

STORED 

STORIES 

STORY 

STRUCTURE 

STRUCTURED 

STRiXITURCS 

STUDIES 

SUBJECT 

SUBJECTS 

SUBPHOBlEMS 

SUGSELIT.I 

SUBSYSTEM 

SUMFX 

SUMMARIES 

SUMMARY 

SU.NG 

SUNSHINE 


(-.0)  S - (P.O)  IV  (►  (-.01.-)  (0,0) 

(-.0)  S - (P  R.Pfi  («.())'  AQ  EL3 

(-.0)  Et(A  S M2  ERR  lAYl.O)  AVC’  (AVR.O) 

(-.0)  S - T ACS  N F £H  (.-  (-.0).-)  (0,0) 

(-.0)  S - T (EVL.O)  EVC-  (EVR.O)  ((-  (-.0),-;41)  (T.O).DX) 

(-.0)  S - T IH3-  ER  ;V2  OW 
(-.0)  S - T IV>  V (F.O) 

(-.0)  S - T IH3  (-  (-.0).-)  \K.O)  AEO'  S - T IH3  (►  (.,0),-i«;)  (K.O) 

(-.0)  S - T AC  (►  (-.0).-;');)  (K.O) 

(.  0)  S - T AA'  (-  (-.0).-)  (P.O) 

C-,0)  S - T UW4'  EH  iH2  (.-  (-.0)  -.01  (7H  (SH.O).  Sm) 

(-.0)  S - T (AAfi.AO)  C«'  {-  <-,0),-)  (0,0) 

(-,C)  S - T (AAfl.AO)  CR'  lY  (Z;4;.'7,0)  S) 

(-.0)  S - T AC  ER  iV 

(-.0)  S - DR  H tH3  (-  (-.0).-)  Shi. 81  ER 

(-.0)  S - OR  R EH3-  (-  (-.0),-)  SH'.g;  tH  V-  (-.0).-)  (O.OKO) 

(-,())  S - DR  H Efi3'  ( ,0).-)  SHI.8I  ER  (ZIAI.iZ.O)  S) 

(-.0)  S - T uHO:  ((►  (-.0).-)  O.OXj  IY  (Z;4;.(Z,0)  S) 

(-.0)  S M<  - (-.0).-)  SH  1h3  ((-  (-.0),-!4;)  (T.O).OX) 

(-.0)  D AV  (►  (-.0).-)  SH  i!)3  (-  (-.0),-)  S (HH.O) 

(-.0)  S UH2  (.-  (-,0).-i  ^p  H.PB  '.a, OH  AO'  (-  (-.0),-;«;)  (B,0)  El?  M (Zi'AI.CZ.OI  S) 
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(-.0)  s UHv  (►  (-.o).-;a.i  (Ei.o)  s ihA' s - t (Ihc  m,em) 

(-.0)  S Ull2'  M EH2  i>-  (-.0),-)  S <HH,0) 

(-.0)  S UHA'  !,(  P IV2  LZ:4;.(Z.0l  S) 

(-.0)  S UHA'  M P IV2 
(-.0)  S Ul)/;  l;x 

(-.0)  S UH2  N Sri  (AYL.O)  AYC  (AVR.O)  (N  DX) 
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Appendix  111-C-l.  AI  Retrieval  Languaf,e  Grammar:  AIXF 
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COMPUTATIONAL  LINGUISTICS 
UNOEHSTANOING 
AOAT’TATION 
INTERACTIVE  DESIGN 
DESIGN 

AUTOMATIC  PROGHAMMiNG 
HYPOTHESIS  rOPMATiON 
DEDUCVIVE  retrieval 
GEOMETRIC  MODELING 

interactive  knowledge  systems 

COGNITIVE  SCIENCE 
COGNITION 
AUTOMATION 
DATA  STRUCIURi  A 
formal  ScMANTiCS 
LANUilAGE  L'NIHvT.TANOiNG 
<SMTNij'Mri;u>  . cTOPic.'s. 

TOPIC  tMFNU/S> 

NlJ/S> 

<SUBJ1  (;T/Ii> 

<AH(  A/G> 

<JMAV>  > CAN 
COUl  0 
SHOULD 
MUST 
MAY 
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<si>  • I 

WE 

<JWHAT!QE>  . <JWKAT'WHICH>  <JBE  > 
WHAfS 

<JDATE>  . tJDATfli 

fHE  LAST  <5TIM£S» 

<tOATF)>  <$CONJllh»CTION>  <:SDAT£1> 
<SDATfl>  «-{TH80UCH><tOAT£l> 
<SOATE  1 > - <tVEAR> 
cJMONTH- 

THE  <MONTH/S>  OF  <tMONT‘l> 
fJMONTH>  <$yEAB> 

<SNUMII|;R>  . <tHljN08C0S> 

<SHIjN(lRCOS>  <SNUMBERl> 

<STIMES>  - l.«)NTHS 

issues 

VOLUMES 

VEARS 

TIMES 

<iTHROLlGH>  . TO 
THROUGH 
TILL 

<$V£AR>  . NIKETELN  <JHUMaERl> 

<tMOMTH>  . MAV 
JANUARY 
EeBRUARY 
MARCH 
APRIL 
JUNE 
JULY 
AUGUST 
SEl'TfMBER 
OCTOOER 
NOVEMBER 
DECEMBER 

<MONrH/S>  • MONTHS 
MONTH 

<$DESIRF>  . <SWANT> 

WOULD  like 
DESIRE 

<JWANT>  . DESIRE 
SEEi: 

WANT 

WISH 

<$Dir.ITS>  . ONE 
TWO 
THREE 
TOUR 
f IVE 
SIX 
SEVEN 
t iC-HT 
NINE 
<tDO>  . DO 
DOES 
DIO 

<JOON'T,  . DON’T 
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Dinfj'T 
DOESH'T 
<sr>0>  NOT 

<£no'SOMi'>  • ctno'>  <ssowli> 

<SFtL!:>  . f ILE 
COPV 

<JFINI5MCl)^  . «iriN]5MtDl> 

ALL  <$r!NlSHE0l> 

<SFINISHE1>I>  - THSOUGH 
DONE 
FINISHED 
<SFROM>  . IN 
OF 

FROM 

AMONG 

<JGETIAFFIL!AT!CN>  • WHLHE  <SOO>  <STHEV>  WORK 
<SOQia  <$THEV>  WORK  <iWHERE> 

<SWHAT'BE>  <;tTritlR>  <$AFHLIAT10N> 

WHAT  <ADORESS/S>  ^i8£>  GIVEN  FOR  <JlTS'AUTHOfi> 

<5THEV>  . THEV 
HE 
SHE 

<SWHEHLS  '.SWHEHI  1> 

<SWHLHEl>  .:JCONJUVCTiON>  <JWHta£l> 

<STHEfR>  • THEIR 
HIS 
HER 

cJITSiAUTHORv  - <i!TS>  <.AijTHOR/S> 

THE.  <AilTHOR/S>  <SCriTHATiPIEC£> 

<SG£I'AUThOR>  - <£PROVIO£>  <iITS'AUTriOR> 

WHO 

WHO  WROTE  <STHATiPlECE> 

WHO  <£nE>  v;ns'AUTnoR> 

<£PROV!nc^  V vsr.iMMi  > 

<jwHATini:> 

<£ThaT'PU(:e>  r <sTHi:;;i  > 

<STHATH‘IECE2> 

<$r.LV'AUTHOR'nAU>  . «SPW0V1DC>  ^SiTS  AUTHOS'OATEi 
<SITS'AUTnOR'nAlE>  . <$ITS>  . SAUTiKlRS’OATt > 

THE  tSAUIHURSTlAU  > <50FH  h.'\T‘PtECE> 

«jr.tTH)AU  > . <$H«r)ViDE>  <SiTS'OAIE> 

<SWHtN>  ^.;WHAT ''WHICH H <N,v^NTH/Sv  <$DE[PAST)>  <JTHAT'P1ECE>  c$WHITTENl> 
V/IU.N  <tnr^  <JTHAT'PlECt>  <tWRlTTENl> 

<£ITS‘UAtf>  . <S!TS>  <r)ATE/S> 

the  <0ATE/S>  ^$Of‘rMAT'PJEC£> 

<$WHtN-  . <SrHOM> 

SIAiCL 

AFTER 

BEFORE 

D'-HING 

<8WH1TTEN1>  .PUBLiSHEt) 

PRINTED 
WRITTEN 
whittl n up 
1 5 5 Lj  E D 
SEi  EASED 
PRODUCED 

<$WHITTEN;  2^  . PUBLIS'  EO 
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PRlNTtli 
WttlTTfN 
V/WITTfN  UP 
ISSUED 
fin  1 AGCO 
PWOOUCEO 

■ctr,ETi)NfO>  - <ir.CI‘ArFlLIATION> 

<St:EViAUT»OP» 

<SGEViauTUORiOAT£> 

<Sr.ETi«AIE> 

<;nETiNtwisi> 

<SnET*lUl.E> 

cSf;ET'llTLEiUKWtSf> 

<5r.ET'inLE'OLI)KSI> 

<St;E1'PU8LlSH£P> 

<SCET>«EKHEK'CES> 

<5GET‘OLC;EST> 

<$CETiNEWLSr>  - <SPflOVlDE>  THE  <SMEV.'EST> 

<SGET'nTLE>  . <SPROV13£>  «:JITS'TmE> 

WHICH  *OHE/S> 

<$GETillTLEiHEW£SI>  - <SG£r>nTLE>  OF  THE  <JN£WEST> 
<$GETiT1TLE'OL:>EST>  . <iGET'TITLE>  of  THE  <JOLDEST> 
<SG£TiPi;BLlSHER>  - <SP«OV10£>  <JlTS>PUBLlSH£fi> 

<SnE>  <STHATPI£CE>  PUBLISHED  <JIN>  cSS0U8CEZ> 
WHERE  DID  <JThAT‘PIEC£>  APPEAR 
<$G£7iRtEEHEK'CES'  - <JPROV10E>  <SITS'«Ef £BEACES> 

WHO  <SB£!PAST]>  <iCnEO>  <tlM>  <tTHAT!PIEC£> 

<SDO'>  <STHAT'PIECE>  <tCIT£>  <SS0UfiCE2> 
<JHOW<WAW;.  »£FE«EfvCES  <SWE«EirHE8£i;wTHATiPI£C£» 
<$r.£T'OLOEST>  . <JPfiOViOE>  THE  <$OLOEST> 

<$NEWEST>  . <$HEWFST1> 

<JNEWESTI  > ONES 
<SNEWfSTl>  <JNUMnrR> 

<SNEV/ESTl>  <{NlJMnEH>  <$PIEC£SI> 

<$N£wrSTl>  <jrRO»H>  <STHAT'P1£CE> 

<SN£WFST1>  ^SPIECESl  2> 

<$0L0£sr>  . <50Ln£sri> 

!$OLnFSTl>  ONES 
<SOLnESTl>  <SNUMnER> 

<JOLnESTl>  ^SrHOM>  <tTHATiPIEC£> 

<JITS'PUBL1SH£R^  . <S!TS>  <PUBLISHEH/S> 

THE  <PUBllSHfR/S>  <$0F'1HAT’P!EC£> 

<SIN>  . IN 
OV 

<$SOUHCE2>  ^ <$r,OEJf  FKi  NCt3> 

<$SOURCt^ 

<SA>  <JS0URCE> 

<SA>  <tRrCENT>  <SSOURCE> 

<SITS'RLI  I.UIToCtS^  ■ <JlTS>  <REEEHEh!CE/S> 

THE  <REEI;hKI»-CI/?>  ^SFR0M>  <JTHATiPuC£> 
cSANV'PIECES'  'iCiTEO>  <StN>  <STHAT'PIEC£> 
<SM0W'LmN'7'>  . <SWi'lAT‘WHlCh> 

HOW  MANV 

<JWE«e:H  Ilf  Hi  'IN  rnnTiPlECE>  - <$B£'THE«t> 
cSBEj  given 

cSBElTEIfRl  j>  <SFWDW>  «JTHAT'PIECE> 

<:SITS'T!TLE>  . cJITSj  <T1TLE/S> 

the  <Ti:l£/S>  <$0E'THA7'PIECE> 
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<ON£/S>  . ONE 
ONES 

cSIWAKNA^  - <$I  0>  LIKE 
<SI>  <snEsiRr> 

<SLFMMt  > . I FT  <SME:. 

ins 

<SMAV>  <S!> 

<$WOlJLn>  r WOULD 
CAN 
COULD 
<tllST,  . LIST 
PRINT 
TRAN'SWi  I 
WHITE 

<jaivt>  . <Sr.lVEl> 

GET  TOR 
TELL 
<tMr>  ' ML, 

US 

<JGIVE1-'  • GET 
GIVE 
SHOW 

<tCRIPE>  - cSlIE-  <iHi  ARSAY>  <v'>LWAVS>  <iSLOW> 

HAVEN'T  '.'OU  TiNlSHED 
WHV  <Sni  > <<.i'IARSAV>  SO  SLO'W 
DO  HLSI'O'.SLS  tVEH  COME  FASTER 
HOV/  ^SN-AV,  ^Si>  <SIMPR0V£'HS' 

DO  ALL  OUERitS  TAKE  THIS  lOKC 

HOW  LONG  DOES  IT  take 

WHEN  WILL  <SHrARSAV>  HAVE  THE  ANSWER 

DOES  IT  ALV/AVS  TAKE  THIS  LONG  TO  ANSVYER  cSME> 

WHAT  'tMAy>  <SI>  00  70  <SIMP»0V£<HS> 

<SSLOW>  . SO  SLOW 
SLOW 
THIS  SLOW 

<J1MPH0VE‘HS>  . MEi  l‘ 

SPEED  .-$1(1  AnSAV>  UP 
HELP  <1HEAR5AV> 

USE  <SHrARSAY>  EEEICIEMLY 
ctHEl-P,.  ■■  il!:i  P 

HOW  DIG  IS  the  DATA  BASE 

<SWUAT50HTS'Cr>  ^HrTKlEVAi/S>  CAN  4MEAS  AY>  DO 
tell  <sme^  what  TO  DO 

<SWHAT'WHICE|-  .JMi  NiJ'MI  NU»  <$MAY>  <$!>  <S5ErK> 
<SVv'HAISORlS>OF>  KilHItVAl  <KEV/S>  <SMAV>  <J1>  <;SEEK> 
<SWHAT'SORTS'OE>  ^cPIECESi^  '$f)E;PRC5)>  AVAILABLE 
<iWHATiS08TSOE>  cJME  NU'MENU>  <snE>  STORED 
WHAT  IG  KNOWN  EVE'HV  <SPIECtSl> 

WHAT  no  <SI>  HAVE  10  DO 
CAN  VOiJ  HEi  P 

vSWHAI'SORTS'Of:  <SE>^ENU'MENU>  <JBE''.HEHE> 

CAN  YOU  Hi  I P <$ME> 

HELP  <iME> 

<SPH0V1CE>  <SA>  viMENiJMlNU> 

<tWHAT'IS>  ^SSOME>  vtWFNlJ'MEUU>  <SFROM>  <J«1> 
<SWHAfiWHlCH>  FACTS  ARE  STORED 
,SWHAT':S>  THE  SIZE  OF  <$hEARSAV, 

WHAT  :JMAY>  <SI>  <JASK> 
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WHA!  CAN  <tHrARSAV>  DO 
ciWKAl 'SOh  1 5*0"  > » fv*M>WKiCM>  <iiOflTS>  Of 
./Al/S>  • KtriiltVAi 
^$SEfK.  - BEQUtST 
CHOOSE 
SEE*: 

<KEy/S>  r ‘-EVS 

KEV 

^^RE>  ■■  - 

<tWHiCH>  <JMf.NTION> 

-tWHATlS-  . WHAT'S 

v.'HAT  .snr|PRrsi> 

<JSOMF>  . .:SSnMFl> 

i>  OE  iHf 

<SHOW'M/Ilo' ALTi)ORS  . . ^SMOW  MfilvY,  <Aii7H0fi/S> 
<jnO'SOM('>  <A!ir^iOS/S^ 

^S'iOWU.ffil.-V  nc  . . SHrAVH.(.',KY'OF‘:il|  M>  <js£> 

.SP.i'  1 . <S  OMKlE'!»lHH> 

. ^tUDW'T/Ar.H', 

<iliDWH,UiKV>  or  <SIKAT'Pl£Ct3> 

<SSor.u  ■<>►'; 

<SSGM(  1>  Of  <i!AAM'p;LCtA> 

.STHAT'I'iE(:K3>  • <4TiiAT'PiEC£2> 

<4TAI  M,- 

<iHOW‘W''i'>'V'Pi:c(  • <si‘Ov/-MnivY'r;ccES?> 

<ino'>  fS/AY'Piicfs, 

^JHOy.'iMAlYV'r.ObECETiECES-  - <SD<><>  <SAK-Y-&CueC£'PiECtE.> 
<?liOVf  l.!A)vV';>;ECf  firsOMf  .iS0u8Cf2> 
<SW1-0.T'WH!CH»  -SSOUHCiv 
^SitOW  W.NV>  fiSObRC£'P;tCtS> 

<4S0L>RCt  > • <$A1>  jOUSNAL 

ASSOCIATIOM  FO?  COMPUTATiOKAL  ‘JNGUJSTICS 

COGNITIVE  PSVCHOlOGY 

ACE 

A1  TEXT 

ARPA  SURNOTES 
ilGART  NEY/SEETUh 
COMl.AiMCATlONS  Of  THE  AC W 
CACM 

COMPUTII.C  -.SURVEV/S> 

CO'X'PiJTIAG  REA';E'\'*'j 
Jf'.TORMA:  AON  ANil  CON'TROi 
*LEf  T RAA'Cf-C. T IC 
l.iCAI  fPRCCf  I 

irip  <r»ccri ciNG/S' 

.kmjRNAI  or  T Ht  A(.M 
<$fiT>NE)RE OS > - <5N(jMH.'Hl>  HONURCO 
A HuNDRCD 

cSNUMM:  R 1 > . <snir.iTS> 

<STE  ( NS> 
i 7iE > •-  v'S |■M> 

<srvf.^  OELN 

<S T M>  r I AM 

I'M 
WC'Ri 
WE  ARi. 

<SI'VE>  - i hAVC 
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WtVt 
Wt  HAVi 

<(rL)>  » <si>  WO1-K.0 

ro 

WiV 
<SITS>  . THE 
THtlR 

ITS 

<SOFiIHATiPUCf  ■.  . Of  <iTn.MiPlECE> 
fWOW  cSTHAT'PUCt> 

<PuaLiSHE(}/5'^  • PuawsHEiis 
PLi3LlSHE« 

*R£ff  (iK.'CL/S>  . REFEKf  lcCt 
PEi't  KCIvC.rS 
<TJTLt/S>  . titles 
TITI.L 

<wosn/s^  . wo»ns 

WORD 

<PHRAS(/S>  . PilWASES 
PKBASC 

<SIAgt>  - las: 

MOST  RrCtNTLV 
<tirARf\!ll>'0:-  . LIAQMt.o 

GRAMMATICAS  IWFLHr.I.CE 
MtllPAl  NtlWOMKS 
AfiSTL'AcnOW 
OVSAl.CC  CLliSTiR;>j.S 
Cf.U  ASSfMBLS'  THI  ORS' 

<JI!ST'1HLM;.  . TMi;  <JNIW{;ST> 

<SThr,'BFXT> 

> ,i:H('IJEXT> 

cSMAV>  <Ji>  HAVE  <STHATipiLCE>  <U!STEO> 
<Si!ST  . ThE  ..JNrV,'FSr> 

<SliST  . <srhAT'PiECE> 

<$THE-'(i  XT>  • IHi  MXT 

Tni  N'fXT  <l(«ijMHER' 

TnE  FifiSr 

<jf  TO  cSNUMiit:f.’> 

BETwiiu  >;MjMI!ER>  and  <sk;l'mber>  of  Them 

MOKE 

The  TlRf-T  <ShiuMBER> 

<$L)ST‘0^  , ilSTEO 
PSl^Tk;) 

WKl  T T t fv 

cSlISTiA'C-*  • Ll5T]fiG 
PR  1 NT  iN‘G 
TKANSMj7Tl!,G 
V/KIT  INC. 

<$MAK£>  . COi  V 
W K i 1 1 
MA:;i 
PkCOoCE 

CtNf 

<SMAKL'AI)Lt^  ' 

<$MAKt‘A'?  ;LE  > OT  ikiE  <SNEWCG’i> 
vt'XANE,  A i'^C  > 

<SMAKC‘[  tl.L'»  ' vJMAKE'A'i  ILE> 

<SMAKE  A*'  iLk:>  <srKOM>  <$ThAT 'PIECE > 
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<SKF1  . * S!"'  '\7lO  ;o 
ABOlj: 
r;':or'^kAiuG 
ON 

H(  ( i.:N;:jii  lO 
n!?,CuSSifwC 
CONCtWNJ.'.G 
NI  IJ  HONING 

<DISCU5'3/S>  . DiSCUSStS 
DISCUSS 

ciMFN'flGN:D>  - :SnE>  <tRE> 

<?■,<!  :J  ':ion'LO,-have> 

<iMFNTjONEr  - MAVt>  - <tMf  NT ir/N£G;PP]> 

SFLATFD  TO 
CONCFSNtO 

<Mi-NTIOK'/S>  - MFOnONS 
MU')  HON 

<SW»ITTEN.  - ^SWWlTTtNU 

<$Prr.TNTi.V>  .$WRITTtNl> 

<swGii  eNi>  <sr;t.fntl¥> 

<$W«!TTE’J  7>  . ^SWRITTEM  ?> 

<SRrr.PNfLY  2>  tS'vVRlT'tM  2> 

<CWP.  TtN'I  2>  <iRrCi.NHV  2> 

<TOr'IC/S>  - TOPICS 
TOPIC 

<MftVijyS>  • MFKIjl' 

Mt  (Vi J 

<SlJD.Ii,(:T  . SuBjCCTS 

sunjicr 

cARI  A/S;.  • ARCAS 
AR(  A 

<SWV ^ *■  C'UR 

<JN>:CSTAIfMi  NH  - <Sli:iWAM>  TO  <tACQUIRE>  <tWHAT> 
<jnoN  T>  <$c;ivt>  <jwha7> 

<SUN'.VANT>  V WOUlO  NOT  l,Kt 
.SnO.N'T>  <JWAKT> 

<s;vFa'IS11>  lAGT 
N-w(sr 

<SIAS'>  -:kV.'Wi:TtN> 

lATSSi 

MOST  RrCENt 
<GiNO>  • I.'O 

JMCf.R2-*  ■■■  <STEN'5'' 

<STPNS-.  ^5ITIwITS> 

<STtf  !,S>  . N!lJEIt.tN 
UN 

tLCVM) 

tvti.:.  o: 

T i 0 T t i V 
FOUR  i'lF  N' 
riFTFfN 
SiNTFFM 
SFVtP.'FEF 
EIGHTH  .. 

<jTEN’‘''^  » TW'Ff^ITV 
, .IRTY 
FCRTV 
• ! T T V 


SIXTY 
St  Vt  NCV 
EK^HTY 
N'TN'ETY 

<SOLOESTi>  - OLDEST 
FIRST 
EARi  lEST 

FIRST  <JWR1TTlN> 

<tOUHS!  i Y't  S>  . <HCIH1EVAL/S> 
<SMV>  ATTENTlOrj 
WVSEir 
OURSELVtS 
ALL  ■:RE^Wi£VAL/S> 
<$Y/HE'M)ATt  . - THIS  YEAR 

last  year 
SITYCE  last  year 
<JWhEN'>  ■SnATE> 
<STORYy5:.  - STORIES 
STORY 

<ART1CLE/S>  . ARTICLES 
ARTICLE 

<BOOK/S>  • BOOKS 
BOOK 

<PAPE«/S>  • PAPERS 
PAPER 

cABSTRACT/'.^  - ABSTRACTS 

ABSTRACT 

<RROCt:i  |)jKG/S>  • PROCLE'ilKGS 
PHOCLi.ltilvO 
<REPOPT/S>  ' Ri;PORIS 
REPORT 

<ISSUF./S>  * ISSUES 
ISSUE 

<JOiJfiNAL/S>  - journals 
jOUSNAi 

^REVIEVZ/S^  . Rl  VIlV/S 
<VO;  UI.M  .'S>  . VOLtiMf  S 
<SURVFVyS>  - SURVLYS 
SURVt Y 

<SUMM.A«V/S>  " SUMMARIES 
SUMSTARY 

<STORY/S  ■ STORIES 
STORY 

<ART]ClE./Y  P>  ' ASTiCLES 

article 

<BOOK/S  2>  • BOOKS 
BOOK 

<f’APEW/S  2>  . PAPERS 
PAPER 

<abstra::t,'S  2>  • abstracts 

ABSTRACT 

tPROCEl  DING/S  2>  • PRGCLEDiNGS 
PROCt  LDINC 

»REPORT/S  2>  . REPORTS 
REPORT 

<1SSUE/S  2>  • ISSUES 

ISSUE 

<jOJHNAE/S  2>  - jOuRNAlS 
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JOURNAL 

<Rrv)[;w/5  ?>  - HrviLws 
<VOLlJMr/S  7.>  . VOLUMCS 
<SURVKV/!i  2>  . SURVtVS 
SURVFV 

<SUMMARY/G  2>  . SUMMARIES 
SUMMARY 

<JP0L1TENESS>  • PLLASC 
THANKS 
THANX  YOU 

<JPHUNLil)AIC  > - <tirMMK>  *SfifSVHlCr>  <JOUfiSELVtS>  TO  <SANViPIECES> 

<$irMME>  ^iCrSV«lCT>  <JOURGClVES>  TO  <$ANViUOOATEiPlECES>  <tWRITTENl>  <$V/HEN!DATE> 
<SI'M>  TNTEHESTEl)  IN  <tANY'UOOATE'PlEC£S>  <JWRirrENl>  <JWHEN>0AT£> 

<JCHOOS£>  <$FROM>  <SANY';.'ODAT£iPlEC£S>  ctWHENiDAT£> 

<JRESTHK;T.v  . CONFINE 
RESTRICT 
LIMIT 

<tPRUNE'i  1GT>  ■ <SPRUNE'!)AT£> 

<$pRUNriiir.TiroPic> 

<SPRUN£ii  15TiAUTHOR> 

<SPRUNtillGT'rnL£> 

<SPfiUNtil  1GTICITAT10N> 

<JPRUMEi|  lGTiUAIE> 

<JPRUNE'IIGTiGOURCE^ 

<JPRUNEil  lGTiS0U«CE'0AT£> 

<SPRijNEi|  igtiwhh;eiwritten> 

cSPflUNC'l  ir.TiIOPIC>  - <?00>  <SSOMEWi:hEM>  <JALSO'MENTIONiTOPICS> 

<SrM>  ONLY  INTERESTED  IN  <JWHAT> 

<SnE>  <JTHAT'P1ECE>  <tHr'rOPICS> 

<SRE>  <SGOMf  iOE'THt:M>  <tAlGO>  <tSr'TOPlCS> 

<SHOY/'M/\N'VK)F‘ntl  M>  <t A1  SO'MEiniON'TOPICS> 

<SPRUNrn  igtial)tmor>  . <snr.^  <sr.OM(:wvMi:M>  <jwritt£N'Dy>  «:Jauthors> 

<JI!OS«/'MAtv>'inL.  c;WRlTTEN  HV>  <JAUTHORS> 

<snE>  <sr.0Mf:>  nv  <tAUTnoRS> 

<SPRIJNi;il  IGTirrLE>  • <SWHATWh!CH>  <T1TLI/S>  <SMFST10N>  <STOPICS> 

<JPRUNi:'(  IGT :C11AT10N>  . <$nO'>  <$S0ME>0HVHI:M>  4CIT£>  <SAUTHORS> 

<$nr',  ^SAUTEIORG.  tjCITEO>  <$!N>  <STHATiPi£CE> 
cSHOWMANYSTr'iltlM’  fLClTE>  <SAUTf!ORS> 
cSnF'>  <$AUTEiORS>  <$CITE()>  <SiN>  <$SOME'<)E*l  HEM> 
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Appendix  Ill-C~2  A1  Retrieval  Language  Grammar;  A1X15 
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CAN  YOU  HFtP  ME 

CHOOSE  AMONC  SJOLRiA;  .f  . 8EE0RF  <jOATt> 

CURING  Wi'AT  MOKThiS -£‘,VtKL  - THeV  <£WRiT  TE  N'rOBt  IShED, 

GENERATE  A COPY  OF  -KTSE 

MAS  <SAUThCK'S-  <>\\K;TTEM'ij  USHEC,  <SANvr-APEi<S-.  clTHISVtAR, 

HAS  cSAUTUOR 'S  . ,JYR;:7FN  Pi/J,  iSHEO,  ANYTHING  <$;ATELV> 

HAS  -SA'JTL'OR  S,-  nrc.  Pi T|  ii;'SCEU  IN'  -JANYPAPCRS, 

HASNT  <5APArLK-  cCAilOiiT  TOPIC-  GEH/KFilASEO 
HASN'T  -S70PICS-.  ilF  N ;)  j;;  <SjOi;H.NA'i> 

HAVE  - JANA' PAf-liVS'-'EiT/A-. Te.gr-  APPI  ARFO 

ilAVE  -JANVPAi'!  R;.-A.'’PF.T'KLi)  -JAIlOi.r  TOPIC- 

iiAVE  < JAiji eiOR ''I  . -i Wh IT Tt  NY'iiBLlSHtO,  <JTHiSYEAR> 

HAVEN  ! VCAJ  f iNiSri  i! 

he;  P 

HOW  HIG  IS  (Eil  UAA  HAS! 

tiOW  CAN  I U:.i  .'..iSVS'IM  iFtiCIENTuV 

HOW  i ONG  -jnOES  i VAKL- 

I'l)  LIKE  TO  KNOW  Til  < JAND'OR  AuTEiOR'DATEii ITiE,  OF  <JTHE  PAPER- 
LIST  ,JOuA'-T,IV,  JNDBIO 

LIST  nnwIIN  -JOEANiilY:  AN,)  -JOUANTIIV,  OF  THEM 
LIST  TeiE  -JPAPERS  -tfiYAUTEiOR, 
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AlXiS 


NO  ThANKi 
OK 

Pi  I ACC  till  I-  Mt 

PLIAGC  LiCT  ^STHI  Abii-IOSS' 

pli;a5E  make  Mt  A filc  or  TnosE: 

PWINT  cS0UAN’w:v> 

pROOucr  A corv  or  <50iJAM!iv>  <spap£hs> 

SEl  lCI  FROM  <,<:PAPERS>  ^SABOUT  TCPIC> 

SHOW  Mf  <:S00A;vilTV> 

SHOW  r/F  iVS  ^SANO-OR'A'JTHOP'OATS'TiTLtj 
SUBSELl.Oi  FkOM  .trOPK;S> 

SURE  thanks 

TELL  KFK  ^SWHAi  TonO^ 

TELL  Mt  IHE  -rSAI.O'OR  AuT-iOS'OATE'T'.TLtv  or  <jauAN7ITY> 

THANK  YOU  .SWi'iii  .■  OONt 
TRANSW!(  '<.0i;ANTJTV> 

WHAT  cfAfiOi:' TOPIC.-. 

WHAT  <f.f:AN  ! [!0>  TO  SPi.Ei)  VO.;  i;P 
WHAT  <SnO  itiAVL  TOi!0> 

WHAT  <f,IS  ■ i fS  <EAiC009'AoTt.riTi  :',ATt'TlTi.L.. 

WHAT  . S.IOUPNAi  .’p.-.  OHRING  <pOATr>  AN,)  ittlATf^  NT  ION  TOPiC  > 

WHAT  .tPAPiPK.  .>w!n:;:;.vtcp:c> 

WHAT  <tSORF0f  SCMVAWY.-  IS  AVANAJ.i 
VS'HAT  ADOP!!;!;  if.  GIVEN  TOR  .JTfn  AO'THCSS. 

WHAT  AP/OREV.  i.',;  ARi  GiVi  !,  f OR  ^STht  AUTt!ORS> 

WHAT  CAN  ..STHi  system OOv 

WHAT  CON)  ( PI  NCI  WAS  AT  <SWGnKPlACE>  OR  AT  vSr.EOPLAC£> 

WHAT  COM  I Pi.NCi  WAS  AT  ^SWCiRK?;. ACC-  OR  AT  .$W0RKPLAC£> 
WHAT  rACi:-:  .•r.A.’H  . SiOKI  1) 

WHAT  Ki  Y \YOR<5  Rl  tATL'S  10  .■ST0P:CS> 

WHAT  Ki  Y V/ORi';.j  StiOUlO  I USE  - Ok  ^S70P:CS> 

W.IAT  KIND  Oi  Ml  :.,JS  ^SARi  thit;i:-. 

WHAI  KINDS  or  SClijiCIS  <{A.Ri  . STORCO 
WHAT  MOST  : ASN 
WHAT  ShOiND  I ASK 
WHAT  SmOLuD  I SAY 

WH'AT  SORTS  OP  .TTOPiCS--  . <iKirNT10,\tO> 

WHAT  si.Hj!  CT  ;;a‘.  ; );i.i)i,is; 

WHAT  TOPIC  MF'.h  can  I C'-GOr.F 

WHAT  TOPICS  ^SAPl  , KFi  AT,;,  <SM.PIC5' 

WHAT  TYPES  or  -TRI  TRIEVA'.  C>‘N'<fAKSAVr,0> 

WHIN  \VIll  you  .lAVi  TriE  A I.  ’ ■ V.  i A 

Will  PI  .PAN;  . -l:op;cs.  ■.;i(:Nro> 

WH!i:i  .$00  iHi  V Y.'Oi:’. . 

WHtPt.  010  <L;Hi  I*/AT'C;T-  Ai*r'._'>-. 

WHICH  ^SAiTtXl-.  • $CON t AI K: -$,Or‘,CS> 

WHICH  A.LjT  lIORS  ' $S^i  NT  ION  , OPtC  > 

W i-liCH  CONflNI  i.CtS  W iKt  A,  *srfLCKL.ACt>ORA*  .$.»t.vyPlACE> 
WHICH  IS  <iGOANTi;Y> 

WHICH  NOTES  <$AU0L;T  TOPIC  . -$ALSO  ME  UTION  T0PIC> 

WHICH  ONES 

WHICH  SORT  or  -$Kt  TRIEVAi  Ki  v'S>  CAN  I SEEK 
WHICH  TITi  fS  <$MFNT10.N  TOPIC. 

WHICH  WAS  the  1 AST  ARTICLE  <SnVAliTHOR> 

W'O 

'••/  •O  HAS  cf.WL'nTrN'Pb^-vSi-.rP..  -CAROUT  TOPIC. 

WHO  WAS  OUOTEO  IN  <$THE  PAPER. 
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WHO  WAS  THi:  ALlTHO? 

WHO  Wt  Hi  .-i'Hi  AijTHO.<S>  Ot  cSTHf  ?APE«> 
WHOWWOtr  <:SPAHtWS>  .lAtiOOT  10P1C>  <tTH:S  VEAR> 
WHO  WROTE  IT 

WHV  IS  TH(  SYSTEM  :STH!SSiOW> 

WOULD  VOU  LIST  clOUAMlTY> 

WRITE  A ULC  OE  THOSE 
YES  PLEASE 
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<$»rTRIEVALCANHrARr.AYOCi  . <$RrTl’itVALCANi(f  A3SAY>  DO 

<snors  n TAi;r.>  . doe?  it  take 

<jnATFSOF-THr(:ONFfwri,CC>  - DATES  OF  <STrif  CO^f  E«r.liCE/S> 
<S\V'HAT  TODO>  . WHAT  TO  00 
<JCAN I nO>  . CAN  I DO 
c$Tlli;  SYSTEM  (10>  . THE  SYSTEM  DO 
<CtX)  I HAVE  TO  t>0>  . DO  1 HAVE  TO  DO 
<JTHCAiJTHORS>  - THE  AUTHORS 
ANY  AUTHORS 


<$K1LL.  . eCEASr  PfiiNTlNC> 

PLEASE  <CEASEP»INT!NG> 
cCEASE  PRINT1NG>  please 
-.CEA5EPniNTlNG>  - <CEAS£>  <PPINTiNG> 
»CEASE>  - CEASE 
STOP 

TERMINATE 
FINISH 
QUIT 
KILL  the 

<P81NT1NG>  . PRINTING 
LISTING 
TRANSMITTING 
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<5H0WMANV>  . HOW  MANY 
<$HOWN1Ari.’YPAPet<S>  . HOW  MANY  <iPAP£»S 
HOW  MANY  or  ThtSE 

<jAR£THna;^  . <ar£  y;e»e>  there 

<JDOiDIO>  YOU  HAVE 
<Sr)O'0ID>  YOU  HAPPEN  TO  HAVE 
<ARE'WtRE>  ' <$ARE> 

<SWEHE> 

<£WEHE>  • WERE 
WEREN'T 
WERE  NOT 
<S00:D!0>  • DO 
DON'T 
DID 
DIDN'T 

<£DOES'I)l)ESN'T>  . DOES 
DOESN'T 

<SD€7  ME  > • <nET't;iVE>  ME 
<G£I'GlVi.> 

TRY  TO  GET 
TRY  TO  GET  MF 
COUI  n YOU  RETRIEVE 
c1'Y;E>  <Or':MAND'WANT, 

<I'Y/C>  <0£MANO'WANT>  TO  <SEE'i,tT> 
<rD'WE'0>  EiKt  TO  <S£E'i.l;T> 

<GETiGlVE>  . GET 
GIVE 
<IiWE>  . I 
WE 

<DEMAN()iWANT>  . DEMAND 
DESIRE 
WISH 
WANT 

■tfl'O'WE'fi^  • I'D 
WE'O 

<sLt:'(;(;T>  . see 
GET 

<$DON'r(.El  Ml  > . DON  <JG£rME> 

cJWia  HE  IS  ■ • WiirHE  IS 
<SIS^  . IS 
ISIYT 
WAS 
WASN'T 
<$ARE.-  - ARE 
AHi.  NOT 
AW(  N'T 

<SIS  TEIEHI;  - ' -:SI3:.  THEME 
<JH(.R'IMS>  • HEM 
MIS 

<i'WHAT  l!AS>  • WHAT  <hASS1AVC> 
<HAG'HAVE>  . HAS 
HAVE 

<SWhENWAS>  ' WHEN  WAS 
WHEN  WERE 


Tlif-Y  WOW>;>  - <DO'!nE  V>  WOB>; 

<boTHLV>  . <SD0'D1D>  they 

<jDOESuO£SN":>  vHE‘Eil'l.> 

• HE 
SHE 

<$WE'HE  iNTERESVEHiN-  - <SWE'«l'  <IN'':E8LSTEDiN> 
<SWE’HE>  ONLY  JlvTEHi KTED  1N> 

THE  AKi  A ..i'.VE-Ru>  -INTERESTED  IS 
THE  ONLV  AREA  -iWE'RE--.  <1N'£RESTE0IN>  IS 
<LET'S>  <RESTR1CT>  <OURS£LVtS>  TC 
<SWE'HE..  • WE'RE 
WEVE  BEEN' 

WE  HAVE  BEEH 
<1'V> 

<1'M>  - i'r>< 

I AM 

<!NTERESTE(;  iM-.  - INTERESTED  iN 
<irTS>  • lET'S 

lET  <US'ME:- 
<LlS'Mt  ;■  ■ IIS 

<HESTR1CT>  - HESIRICT 
CONE INE 
LIMIT 

<OUR5rt  VES>  . OiiRSEI  VjS 
OUR  ATTENflON 
MYSEL* 

<$WKAT  ABOUT  > • WHAT  AUOU' 

<SWHAT  ARE,  r WHAT  ARE 
<SWHAT  1S>  ' WHAT  !5 
WHAT'S 

<$WH1CH AUTHORS-  - W.'iCH  AilT-'ORS 
<SWH1CH()I  THiE.i  ■ • WHICH  OF  <TntS! 'T  iij  M, 
WHICH  PAI'f  R 
WHICH  -iPAPERS- 
WHICH  <$jOJWNA'i  > 

WHICH  <SJOijRNAi 7S> 

<THE;H,'' Hi  HE-  ' Vhl  SE 
THEM 
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<JWCRt:PlACi>  . CMIJ 
NIH 
NSI 

RIJ'GERS 

SMc; 

SRI 

STANf  OWG 
SUVf  X 

THf  GKA  RrStARC*:  lABS 

THf  lUSTlUl^L  t o?  SEMANTIC  AI.G  COGMTiVE  SIUOiES 
WATSON  Rl  S!  ARC'l 
<5GCOPLAr.:  ^ . fOiN'HURGH 
HAMHUkC. 

Ill  l^ois 
MASOACflijSCTTS 
ROCtllliliK 
SUSSEX 

TH!  SUNSHINE  STATE 
THE  US 
the  USSR 

<JIATEIV>  - lATL'lV 
RECENTLY 
IN  RfCLENT  TiMLS 

<SWRirTEN:E>ljSl  lSHEn>  -WRilTEN 
PUBlISNEO 
hROOuCEU 

<SAEWAvs>  • Always 
reguearly 

USUAL  IV 

<iTHissiow>  . <iHi5'S0'  Slow 
<TH1S'S0>  . THIS 
SO 

»$COMAINEO>  . CONIAINEO 
CONTAINS 

<$SORT  OE  SUMMARV>  . SORT  Of  SuMMARV 
SORTS  Of  SUMMARIES 
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<iALGO  cut  AiiTt;OR..  . AlGO  ^iCITf  AuTriCa> 
<SriT£;  AUTHOR  ■ .:l^  ADUITIOM. 

<JC!Tr  AUTHORS  - - CiTt  s .JAuTm03/5> 

Str'ill  10  <SPAPt:RG>  3V  sSAuTfrtTQVb. 
cCITt  > . CITE 
OUOTE 
iJETmCi.'CE 
DISCUSS 

<INADD1T10N>  - iNACU’TiON 
ALSO 

SlMUl  TAI.'tOUSLV 

<SALS(.'HV  AUTHOR  . . A.SO  . iUV  AiiInOh!s 
<SnVAUTHOR.  :a  ACi)!TiO.\> 

<snv  author.  . -WRilTENyv.  <SAUTH0KTS> 
<W«1TTENRV>  . OY 
WHITTEN  QV 
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<jr!ATt  . . <S'lAR  . 

.Yf  AR. 

cMONIH.-  OF  <Vi  A». 

<M0NFH>  of  <S*<OSVtAK'> 

<M<)NTH> 

<YEAR^  - ^CENTilRV.  .SNijMj!iai-59- 

<Ci:NTURY.  IIUNDREO  iJNijMliCR  l-99> 

<a:NTURV.  liUNlURCD 
<CfcNTURV  . - NiMTtlN 
ElCHTltN 
SEVtN!ttN 
SIXUiN 

<M0NTH>  . iANUAHY 
FtliRUARY 
MARCH 
AHHU 
MAY 
JUNE 

jIjlV 

AUGUST 

SEI’TEMKER 

OCTORER 

NOVEMHKH 

DtCEMHER 

<$TIME'I'I  RKli)'  • <S0UANT!Tv>  «OAVSMi><>l'!HS'VEARS> 

<oays*mo»vths*veaf<s>  • months 
days 

YEARS 


iTHiSY£Afi>  . THIS  YEAR 
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<80UAN’iW,  . iM£  <lAGT>  <SKiU».u)£Ri-99> 
Th£  <lATi:sr>  ‘39> 

THt  <£AIfcSI  > 

<SNkJMHER!-99» 

UP  TO  <JNUMQERl-99> 

<LAST  . EAST 
NEXT 
FIRST 

<LA7£ST  > - earliest 
LATEST 
NEWEST 
OLDEST 
MOST  Rfr.ENT 

, < NUMBER.*  - <$NliWfiEHl-99> 

. V F'UNL.RC()S>  <JNI>MtiEHl-99> 

. < hundreds^  . < 01CIIS>  HUNORtO 

A mmsco 

<(NUMi)FRi-S9>  . < NUMUERR} 
t TEENS* 
c OIGiTS* 

< NUMBERS*  . < ItNS> 

< TENS*  < DIGITS* 

< TEEN'S*  -NINETEEN 

TEN 

ELI  VFN 
TWELVI 

thirteen 

FOURTEEN 

FIFTEEN 

SIXTEEN 

SEVENIiCN 

EiGHTlEN 

< TENS*  • TWENTV 

THIRTY 

FOPTV 

FIFTY 

SIXTY 

SEVENTY 

EIGHTY 

NINETY 

< DIGITS*  - ONE 

TWO 

THRr? 

FOUR 

FIVE 

SIX 

SEVEN 

EIGHT 

NINE 
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<JAND«0«'AUTt!0‘»'nA:i''.  |Ta>  - AuUO^S 

DA'.i*. 

TjTUS 

<ALJTr^'C/*^  '^Tf i FlL'* 

,A":  ^^'nATE'^ILli  .ASDOS  3>  tAulHOR  DAT£'TiTi.E> 
<ALiT^-CR>r)A;L'TUL£.  . AUT^iOR 
DAlt 

Pi;BL1SH£R 

TiTit 

^ANOCR  3>  • AND 
OR 


<SAi.GO  ftiiOul  lOi'iC'  . tis  »$AaOoT  rOi'iC> 

<SAtiOLil  IGi'iC'  ^ ;Ki  AOUiiiCN. 

WHlC**  < ]W  ADDI  f iOf*>  't  nO\  TOAlC  > 

v.'iiiCH  Tone .:nado”;o\> 

<$AGOUT  lOPlOi  - «AGOl’T  > <5TOT‘;CS> 

<WH1CH'1  (tA  I > . W(  MI  lOfJ  TO>'ICi;WHiCn> 

<WHlCMi1nAi>  ■ WHICH 
THAI 

<SA,.50  Ml  i.  Iinv'OriC'  • i!A' AnOI'iOfo  <JMf  NTiONTOPiC> 
< J MI  W T IfKV  TOPIC < ITv  P'OOI > iON'> 

<tMfNiin.MToric^  . <Mrwnf;WTOP;c.’wniCH> 

<SAIiOllT  TOPIC > <SAHt  l!>i  I-i  • 

^MfNTlO.MOnC.-'WiCCHi  • vM(S(:U>M>  <t7o?;cs> 

<SAfiE  Till  m ..  ^tAilOlJT  TCP;C> 

<Aeril!AVt  (Itl  Mi.  <SAoOuT  *OPiC> 

<ABOUT  • AGOlIT 

<MiKT10M:N‘C> 

ON 

<MINiK:N>  - CONCERN 
CONIilOEB 

contain  the  <Pl.«ASE.’G>WOfi).’G> 

OESCSiliE 

niscuss 

DIGCl/SCEb 
Ml  NTiON 
RET  I H TO 
SIT  ATE  TO 

<Mf  NfKi\TA'G  > ' oONCERNjNG 

O.GCUE.r.ING 
Mi  NflOMMC 
Si  GARDiNG 
Si  I I WHING  iO 

<PM»AGs /G'WOPa'G  1 ■ <PHRA5E'\V05r.. 

<PilHA$£  SAM0R:)3> 
cPHRASE'V/OSrii.  . pMpPiSt 
WORO 

<pHfiA5t  .^‘is/ocD3  .■  • Phrases 

WORDS 

<AHC'llAS'L  lii  I >V>  . ARE 
•SAVE  aCE.N 

<SMi  MnC.'.tO  lOPTC'  - ^$»/IMiONT,Ti  .Eio?;cs> 

«SMt  NIiO.vElt.  • ^SMi  IC  IOMJ'n!SCL,-5SEl)> 

HAVE  <SMt  .NT  IONl D'CiSCltSSEI)  > 
last  .SM;  liIIONEGiOISC.j'SSiDi 
WSilTI  N up 

<$MINT!Of.’ED‘;)iSruSSLG>  - MtNTIONEO 
CONCEPNEO 
DlSCo-SSCi) 

REEIWRED  TO 


<JANYPAPt«S'  • <^NV.Mrt^V■SOM^^  ,VP/.PtRS> 

<ak'Y'>via';n  or  ^rHtSfitYjse.' 

<A,WiWAIC>‘C.n;,<t  or  - '^if.SE'lH0Se‘7hF>  <JPA'>ERS> 
<:gAPAPtH>  - A •^I’Ailfcfc/r.i. 

A <Arj  PAPERXC-.  <PAPFf.> 

AN  c>'APERv:V> 

AN  vAt)J,-.PAPFH>:V:.  cPAPtR  . 

..-SACONFEW!  l-rE-  ^PAPEK> 

ANOTHER  <PAPEP> 

ANCTurn  ^Anj:.p;,pf,ii>  <PAPtR> 

<?Ar'EQ-  - <PAPcP:'(;> 
cPape:r,:v> 

<PAPER>:C>  . SOOK 
PAPF(< 

RE'-CPT 

sTc;;v 

<PAP:i</v>  - AnsroACT 
ARTiCU 
SUMMARY 
SURVEY 

<ADJr.PAr'FR>  ^ <ADJ:.l''APEtfV!C> 

<AUJ^;PAPFR:'V> 

<AOJ{^PAT'rR  •'C>  - TECHI.ICAL 
COMPijlp.'G  Siik’Vf  Y 
<Ri  c''nt(:i)»r(:n'> 

<R(  ('.F.NT^•.I1R«1.^^'>  <AOjr;JOL'RMU 
<Anj^J<>iJPJ.Al  70... 

<ACJr.i<)l.iRNAl  7r. . <J0lJRNAi> 

<fii:CF.NriCUR«('NT-  cJXUKNAls 
<jOi;R;.Ai  7<:> 

<NAI.'!  ,.J()|  |R!JAI  'C.v 

^RIT.FNTCUfiRCNr . « AOJ-tjOjRNA'. , cjOUVNMl, 
<A0jr.PAf'l  R'V,  . <A0j'-.JU'jRNA17V,> 

<Af)JT.JOU«NMr'.  ..  v.01URNAL> 

<NAIA(  r jOiJRNAl  '’V> 

<SAi  jCjfiNAL,. 

c»TH|:  PAPER  - • <rhiS'i"AT:TiU  > . JaPEV.' 

<''MlS'riiA  r'THt>  ^AO^i'PAPFR.  .'i'APER'. 

<{Th!;Si;  f'APCRS>  . <lhESl  *i(iOSt>  <;PAPF«Sj 

<JPAPf«A..  . ..pAPfk'S. 
cPAI’FHS'  OS'  Ai 
<An.l"'-'APF«.-  .I’A^EkS-. 
tPAPFWS-  ll.'THl  SiCAKT  aEWSiCTH/. 

. - S.jOi.'PMAL S>  -t^AiiOUI  OPiU> 

<PAPFP:;.  . ASGTkAU.S 
Ate.  )C:i  tS 
nooi;:; 

PAPi  RE, 

R(i'''''r;TS 

?TOP'c;- 

PUMM,',p;r< 

surveys 


cfHiS'TriA7i7(;E>  ' THIS 


'02 


THl 

^THi:SI  i|  H()'5E>  • THLSI. 

those 

<THtS(  'I  IIOS£THL>  . THt 
THlSi: 

those 

<ANS•lWA^'^''SOM:;>  . ANV 
MAk'V 
SOME 
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<JANY  JOURN'Al  S ■ ■ ANY  .S.iOlJfiJjAi.  ;S^ 

ANY  -RfCEMCiJCRI  Mv  <SJ01JRNALS> 

<.$A  journal  > . A ^jnURNAlvC> 

A <JOlJRNAl  ,T>  <0r  THEAnj.(:ONEEaKNCE> 

A fNAfJf,"JOIJflNAL/C> 

1 A <AD.Ir.,IOIJR(;Al  /C>  cJOljRNAU 
. A cADj^-.JOl  iRNAl  , <CCHFi  KEK'CE>  <JOURNAL> 

AN  vJOURNAlVV> 

AN  . JOURNAL  rv>  <0F  TW  Al)J-(:()NFtRLNCE> 

AN  <NAU( !()IJPi.'A17Vj. 

AN  . A;)j, - JOURNAL  ,’V>  .JOURNAL. 

AN  cAnj;:  j()URNAl-;V>  .CONFf  HFNCE  > <J0URNAL> 
.SJOURNAL  - . ..OURNAU 
<JNAM1',>J(HJRNAI. 

.JOURNAL.  . .JOURNAL /C. 

.JOURNAL '.’V. 

.journal  ,’C.  . JOURNAL 
PRlX:[(  OING 
PHOCIIDINGS 
SUMMARY 
TRANLJACTICN 
.JOURNAL  VV>  . ISSUE 

.SNAMl  c JOURNAL  . <NAM(  r-JOURNALVC. 

.NAI^I  ■ JOURNAL  .'V> 

.NFMIo journal  JC.  . CACM 

.JCOGNITIVK  PSVCrlOlOGV> 

COMPUIIN'C  SURVEYS 
SURNOTES 

SICLART  NEWSLLVIER 
COMPlJTIN'G  Rf. VIEWS 
.NAMCr. JOURNAL  7V>  = AfiPA  SURNOTES 
.SiNFORMATIONANOCONTROl. 

.AOJl^  JOURNAl  > • .A0JPJ0URNAL7V> 

.AOjiJOURNAirC  . 

<A0Jr. JOURNAL r’V>  . Ml 
IjCAI 
ACM 
ACL 
lEEl, 

IFIP 

<6THE  journal.  ■ TUi  .SjOURNAuVS. 

THL  <SNAMi  ,;JOI.'RNAL  > 

THE  .mUJ,>JOUWNAL.  <SJOi/RNAl>’S> 

THE  <AOJ;: journal.  .CONFER!  KCt>  <$JOURNAL7S> 

The  .sjournal/s.  .of the aoj-conference. 

.SJOURNAL  /’S.  .of  THt  AOJ-CONFERENC£> 
.SJOlJRNAirS.  . .jOiJRNALX. 

.Rl  CENTCURKlNr.  .JOURNALX. 

.journal  X>  . .JOURNAL. 

.SjOURNALS. 

.SJOURNALS.  • .jOURNALSYV. 

.JOURNALS.'C. 

.JOURNAL S/C>  .journals 
COMMUNICATIONS 
PROCEEDINGS 
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TRAI^'dACTlON’^J 
VOLIJM!  S 

<J0JRNALS/V>  . ISSUES 

<0FTHEA1)J  (;ONM:Kri>Ct>  ’OF  THE  cADjr  J0URNAL> 
OF  THE  <AOJr.jOuS?JAl>  <CONFLHtt«:C£  > 
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<$ACONri  HI  liCI  • AN  sAOJ-CONi ; i-i:KCf/V>  <C:ONFtHi:tYCE  > 

ANOTHI  H <A()JrCOK'n:Hi;iv-CL>  <CONf  EHKI.'Ct  > 

A <8rr.i.NT  AOjCONFt  lil  NCri 

<Rrr,£NT  ADJCONf  IHi  Nets  . <Hi:C£hiT-CUfl»£NT>  <ADJiiCOHf  tHENCE>  <CCMF£H£NC£> 
:Rrr.ENT<;UHHI  u r>  <COIJ(  FHKKCt  > 

, NEXT  i 1N£  SliOULll  8t  <ADJi.CONFrHi:h:C£-/C>  <CONF£H£NC£  > 
cCONFI  HFNCF  » 

<ADJf!CONFtHI  NCI  > - « ADjr^ONF£HtNCtV,V> 

<ADJr-CONF£HtNCLVV>  - A1 
IJCAl 
ACM 
lEIF 
IFU 

<CONF|  HI  NCl  5.  . CONFFHtNCE 
CONVENTION 
MK£i  INC 
SESIilON 

tCONFI  HI  liCLS  > . CONFtHFNCES 
CONVENTIONS 
MEE 1 INCS 
SESSIONS 

<REC£NTCURRENT>  - RECENT 
CURRENT 
NEW 

<SANYCONrEHI  I.CE/S-  • ANY  <SECEN'T  ADjCON'f ERE:nCE/S> 

<R£CENT  AOJCONflHi  ^■CL/S^  - <Ri.CtN'TiCURR£N7>  <AOJ.-CONFE9EN'CE>  <CONFEHENCET!S> 
<R£CtNT‘CU»RFNT>  <CONFt:«ENCC/S> 

<AOjFCONFtHLNC£:.  <CONFEHENCE/S:> 

<CONFEHKK'CE/S'  • <CONFEH£lvC£> 

<CONFEH£NCES  > 

<STH£CONFtHi:NCl/S.'  • THE  <REC£NT  ADJCONFEHENCE/’S> 
ijCAI 
irip 


<5•^'^ORMATiO^■  ANOCOfiT^Ol  > . ;:«OSMATiON  AND  CONTfiO. 

<SCOnNlTIVt  PSVChO.OGV,  -COCNITiVf  PSVChOlOGV 
<$Ai  jOURNAl  . A1  JtUjRNAl 
<$.Al  lc.XT>  » A;  TtXT 

cSTMt  AGr.r,CIATIONfORGOMPljTATIONAi.LlI>:GU:STiCS>  - THt  ASSOCIATION  FOR  COMPUTATIONAL  LINGUISTICS 
cSHf.TWUVALC.ANHf.ARSAV>  . RriMIEVAL  CAN  HLARSAV 
<SH1JMANPR0BLI  l.lSOL'/'i;G>  • HUMAN  PfiOOLLM  solving 
<SRFTH1EVAL  KtVS>  - RETRIEVAL  KEVS 
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<$AUThOS:'S>  ■ -:Aij7tiOR> 

<AljThK)R'  -ANDOR  1>  <AUTriOR> 

<ANa‘OR  I > • AND 
OR 

<AUTHOR>  ^ ALUM  COLLINS 
ALUU  NLWEIL 
AN'N  RUBIN 
AN'THONV  MARTCiLl 
Ai’RlEL  WOStNFI.lt) 

BERNARD  MU  l/’Efl 
BERT  BARilAll 
BUI  WOODS 
E?ONME  NASH  WEBBER 
BRUCE  BUCHANAN 
CARS  UEWltl 
CHHiSTOPliER  RlESHtCK 
CHUCK  RItui  R 
OANNV  BOBiJOW 
DAVE  RUMHIiART 
OAVIO  MARW 
DAVID  MICHIE 
DiCK  SElli^ER 
DONALD  NORMAN 
DOUG  LENAT 
DREW  MCDERMOTT 
ORfVFUC 
EARi.  HUNT 
EARl  SACfWDOri 
ED  f EIGENBAUM 
ED  «IS£E<AN 
[i  i lOT  SOLOWAY 
i • SAkDi;V/AU 
f . NF  CHARNIAK 
FEiCU  NHAUW 
f f I OMAN 
GARY  HI  1,'DPIX 
U.  ORGC  ERN'jT 
GiPS 

HA\“.  CrWT  INI  R 
HARRY  BARROW 
H!  WR  SIMON 
hLRl-ERT  Bl  IlCK 
HILARY  RUfNAM 

huli  and 

HUGH  NAGEL 
:RV  iOBLI 
ISSAC  ASIMOV 
JACK  MINKI  H 
JACK  MOjIu'W 
JAMI  S SlAGU 
Ji  AN  SAMMf  ) 

Jll  I Rl  Y 1'!  I MAN' 

Ji  RWY  I i I DWAN 
JOHN  CASCHNIG 
JOHN  HOLlAND 
JOHN  MCCARTHY 

iQ^iu  hjru^rnwro 
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jO'li  ‘'it  w; 

jliDt.A  PtAf-’i 
kAkI  PlfiijLl 
<1  i t.M  I'BICt 
Kitv  {;;)i  iiv 
Ki'N'  tlAirtTOK 
KiNCi  ?UNG  r U 
LAlJfOl.N  r r.IKLOSSV 
a I I.HMAIJ 
lCON'AHD  LIliP 
Li  i.  ilABNi  ST 

MAO^I  IN't  itAlfS 
MARS'jfv  MASKV 
StARV  Nf«v;<I>RN 
StARV  G'lAV,' 

StlCiiACL  ARRUI 
MK!  RVCt'Kf.ER 
M!‘\5SV 

MiTCtii  l i Iff.WiV 
I'tE  Wt  t [ 

NILS  ,\’iLSSL,N 
MLSGOI'.' 

NOPl SUZuKI 

PAMi  iA  v>;:cosnii(:< 

PAT  'WINSTON 
PERRY  THORNTiVKE 
PETER  Kl/uti. 

RAJ  SEDOV 
RAfWR*  CAiLtR.ii 
RAVMONO  SPRO'JLl 
Ri  i)DV 
RICH  E I K E S 
R!C.‘  SMilH 
RiCf  Sn  l.llCHAlSill 
RiC-  ARD  V/A13INGER 
RICK  liAviS.POTH 
ROUl  WT  REITER 
ROr.i  R SCHA.'W, 

ROM  (Ail  ASDti; 
ROSINFUIl 
sCOTT  I AliLKtAiL 
SEYMOUR  PAPER! 
SiHtOM 

STE  Vt  CO;  i S 
STEVi  RITO 
STES/i  /CtXER 
TEt)  SitORILITEI 
TEWHv  WiE'OLIRAD 
TiiOMAG  MRkGLAMC 
T HOmAG  SYkES 
UtiR 

VIC  Li  SET  il 
WA'.IV  RHOMHERG 
WCXTDS 

WCiOOY  t!,  i !>50i 

voRiCK  w;w;g 

ZOhAR  MAi.MA 
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^Sior'.c:';-  - topics' 

/ViN’OCI'AD'S  ARIiClt 
<ANOK!n'TOPIC5'  . 'TOPIC' 

aOPIC'  'ANOOR  2'  tTOPIC' 

<ANO'<''?  2.'  • ANO 
OR 

<TOPU:'  - A CAI  MOMIOB 

A COMWOW  SFASE  algorithm 
A GAS(f  MOOT  I 
\ I OS  IMG  MOVt 
AMDi.TIlCVH  ORGANIZATIOM 

A PACKI.T  CASfO  APPROACH  TO  KETWOBK  COMMDMCATION 
A PARTIAL  GVAillAlOi; 

A PUOC.iJAM  SVMHi;!;i2r.l)  for  metwors  protocols 
A phooramwik;;  appheniicc 

A PROOl  CHI CKER  FOR  PROTOCOL  TERMlKAT|OM  EXPRESSIONS 
A RADIO  IIvTERVIEV/  OM  SCIENCE  FICTION 
A REGION  ANALVSiS  SUBSVSTtM 
A STlRI  0 I'AIR  Of  V)EV/S 
A TAS':.'  CPICMTiO  DIALOGUE 
A TliALIV.AfLIRGlSr 

A Tf:T0R(  M PROVi  R PLANNING  FOR  PROGRESS 
A Til.fl  IKIMtiN  ANAlVZffi 
A TUTOR  OP  UITOHING  ON  TV 
A TV  PLPCRTt  « 

AH'jTk’ACDCM 

/■OOUISITION  OF  knowledge 
ACTI'vA  KAOWllDGE 
/aT'  Gi  IC  IEOMi.RS 
ADArJTATION 

ADAPTIVE  PRODUCTION  SVSTEMS 
ADVISING  PtIVSiCIANS 
Ai 

a;  lcctu-'-c; 

ALDi  HRAiC  REDoCIlON 

r ; ;.r>L 

! i ilOPITilMiC  AC L.VUi  I ICS 

Ti  L OR  MONt  SOlpTIOMS 

/M  ADAPTIVE  natural  LAMGl^AGE  SYSTEM 

AM  ADSL  MHi  Y ROrjOT 

AN  AXIOMATIC  SYSTEM 

/•MAlOGV  IN  PilCailM  SOLVING 

A’vALY.'iS  Of  CON'TtXT 

ANAIS'GIS  Of  STIi'ENCES 

ART  iriCIAi  iNI‘li  IGENCE 

ASS.'MlLATiOfy  CE  NEW  INfOSMATION 

ASSOCIATIVE  MEMORIES 

ASSOSIATIVL  MFMfTRY 

AUGIPl'NTtO  TRANAITIOS  NETWORKS 

AUTOMATEO  Df  tH.CTlCN 

AliTCl.TATIC  COOING 

AUTOI^ATiC  COMPUTATION 

A.iTOl.-KTTIC  MANTRA  GENERATION 

AUTC'PATIC  PROGRAM  SYNTHES'S  FROM  EXAMPLE  PROBLEMS 
AUTOMATIC  PROGRAM  WHITING 
AliTOMATIC  PROGRAMMING 
A>ilOMAl,l.  PkUOl  Ut  L,ORHlUiNLSs 
AUTOMATIC  theorem  PROVING 
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Ay;ov/.iic 
AXIOK^S  t OK  (.0 
iiAC 

;ii  i !►  ” I I MS 
'j  1 A':  jIK'OV 
i3 lOWf  t ) fiO S‘t 
liKMN  Till  OK'-' 

SL/SiN'l  SS  PKO&lTM  SOLVihG 

CAPTOCtiAPH-.- 

OAOf  3V" IMS 

CAliSAL  Pi  A3;■^v;^JG 

Cl  I 1 A33C'.(H:  V ThI  OKV 

Citi-CyiliG  PROOfS 

CilrSS 

CHI  ‘.li  PTAVIT.G  programs 
ClIiCiJlT  ASAlVSiS 
COuSvT  iC\’ 

COuA'lTlVi  K'CiiOIiC  SVSIcWS 
coGKinvf  sc:f:sc£ 

COMMO.  j Si  i.5E 

COS'fK^iTS;  St.GSc  IWORV  fORMATiON 
COi.ll'i  I H WAVCfOPUS 
('.OiAPljTATtOiVAi.  LiT^GUiSTlCS 
(:0i.G'’ij7cS  .API 

tOi/PiiitP  GASfl)  CONSi.LTASiT 

COi.Ti’iJTt  R GA'.i!)  COMSi-JLIATIOMS 

COSU  IJTEP  COMROUfl)  M/tMPulAIOPS 

CO'Ai'ijrCR  GRAPHICS 

COMi'ljTCR  M;;SiC 

i-Osii'ilT E K Gl  '>V.'(^RKS 

COSTPiGtR  VISION 

OONiirPTuAl  (>i; SCRIPT  10^S 

CONCr  PTU'ii  i'Hl  Ri  ser 

CONfV  PICA:  OV.  V,  AVS 

rONSlRAINT  SATISrACViCN 

CONSlRliCl  H.'G  PWOGRAV.S  FROM  iXAWi'l  lS 

( ONSl  kGCI  ion  Oi  PROGRAMS 

CON'.ixr 

CO.NTINVJOOS  PROCiSSt;. 

C ON ' ROi 

COOPi  RAC'.G  SOURClS  of  KNOWUDGE 

CCPVIN'C.  1 ;5T  STRiiCiURCS 

C L. K v;  l j ciiij!  C I't 

Cv:<!  RNE  uCS 

rvGi  ic 

Tata  bases 

:aTA  basis  for  INTtRACIiVt  DESiGN 

C!A1  A STRuCUiPrS 
L iCiSiON  Till  ORV 
Cf  OGCTiON 

CcOoCTiVi  kETRjEVAL 
CLNOViTiON'A,  'tMANTICS 
CET^TH  PE'iCEPTiON 
CLPiVATiO'Nj  Pi  A.NS 
DESIGN 

DESIGN  AOTOMA'ION 
OrSlGN  IN  THE  ARTS 


OETkCTiOM  O'  liGHT  SOuSCES 

OlOPi  AV  U'hMlN;,LS 

DBAt'.Ori 

Os:v;r.'tj  a cau 

Dvtvfiwic 

OVUAMiC  ClISTEHING 
DYNAMIC  PPOCUAMMiNG 
t.l  I CTUONIC  CIRCUITS 
Li  I CIWONICS 
lN'GUGH 

tVAlilATlON  ruNCTiON’S 
EXPl  Rt  SVMtMS 
explanation  CAPASIlITICS 
EAUllN  OR  FAIRV  TALES 
fi  ATLiPr  tlRIVLN  SYSTEMS 
FIPST  ORDER  LOGIC 
FORMAL  SEMANTICS 
FRAME  TEILORV 
FMAMi  S 

FRAMl-S  AND  IHK  ENVlRONMEwT 

FU/EIV  <K'0WI(DC£ 

fU77y  PWOOIIM  SOLVING 

GAME  OF  I'OKI  R 

GAMf  PLAYiN'G 

GLNIRAL  RljRPOSE  MODELS 

GlN'ERAIION  of  natural  LANGLjACE 

GLOMt  IKK:  MOUELltvG 

GO  OR  GO-MO'.'U 

COAL  GlLK'I.G  components 

GF  AIN  OF  COMPIjIATiON 

GRANFMATICAL  INFERENCE 

graph  intlrprctablE  games 
graph  matching 

Ht  AR$AY 

HETER03TAT1C  THEORY 
HE  UR'S  TIC  PROCRAMUiNG 
HLIJRISriC  TECHNIOOES 
Hill  CLIMRING 
hiPKA!,  liCHAVlOR 
Hi;,‘PA!V  Mi  mORV 
ViSiON 

HVPOItii  :..S  fOPMATION 
IKtAGT  IN’ILnSITV  iiSDERSTANDiNG 
IMi’ROVlr.C  PROGRAMS 
INDuC.TIVi  ^^SMii’iONS 
IVOUSTKiA;  AP.'-'i  .CATION 
::U  KACT  i.'l  PR'  S,  NTATION 
if.’f  L i.’Ci 
INFL  R1  l.-Ci  S 

ll.'L  I RiMlAL  OuTSliON  AN-SWiR.NG 
ORe.eATION  pwcci  ssing  universa 
1\H1  RMARCE  of  PROPERTIES 
INTiLiiCENT  MAC'ilNLS 
INTENTIONS 
INTLRACIIVE  DESIGN 

iNTEKAci  ;vr  K.vDwroGE  systems 
interactive  program  SVNTtiLSIS 
INTERPRETIVE  SEMANTICS 


UvTOliATiON 

IKVARI/M.r.E  FOR  PRODLtM  SOlVIfiG 
INVARIANCES  IN  tME  PERCEPIION  OF  FACES 
IN'VLSIMiNI  ANALYSIS 
nEHAIiON 

knowledge  rased  systems 

KNOWLEIiGE  SYSTEMS 
LAMliOA  CAKX'LUS 
LANGUAGE  COMI’RrHLl.’Sia'J 
LANGUAGE  DESIGN 
LANGUAGE  PARAPtiRASE 
LAI,Cl)AGE  PASCAL 
LANGUAGE  PRIMITIVES 
language  UNDERSTANDING 
LARGE  DATA  BASES 
LrASNIA'G 

LINEAR  If  XlCOMl  TRY 

LOW  ORDERS  Of  RECOGNlf  ION  PERf  OCNAAN'CE 

MACUiNi  INTElilGENCC  IN  Ml  DIf.Al  DIAGNOSIS 

macro  PRIXIE  SSlNG  FOR  AN  ON-'.lNE  NEVYSEEFTEW 

MAtCAGi  Mf  N i IN'  ORMAl  iON  SYSTEMS 

Mi  AN'S  FOR  COWPl.TEC  MOVIES 

Mf  DICAi  COl.SiA  TAIION 

MINil.lAl  Si'ANNlNG  FORE  STS  OR  TREES 

MOTION  IN  SCENE  DESCRIPTION 

neural  NI  TWCRKS 

NE  WSK  I TER  WEPORTERS 

NONDETlRf.tlNISTiC  PROGRAMMING 

OEKIECT  lOCAIiONST  AM)  MOVEMENTS  IN  NATURAL  IMAGES 

OETJICT  MANIPUlAIiNG  ROBOTS 

OPLRAllOMa  RE/.SONINC 

optimal  WOBL!  I.E  SOlVli.G  SEARCH 

OPTlMl/’lf)  CODE  FOR  THt  TRANSFER  OF  COMMENTS 

PAPE  RS  ETV  EtUi.  WOODS 

PABALtI  I i'lf.l  IN  .•■RODLLEY,  SOLVING 

PARTIAL  EVALUATOR 

pattern  DIHICTED  FUNCTION  INVOCATION 
PATTERN  E.UMC'UNG 

pat  te  rn  :;i  coGtciTiON 

f CilCtPTWO'.'S 
E’EIOTOGUAMME  iRV 
E’ICTunC  RECOGEvlTlON 

Planner  lEtEE  l/u»gjages 

PRrOiC'Mc  CAtC.;'^uS 
PREI  ! Ill  N'lS'^.  St  E.'jTNTiCS 
fTilC  E S TUI  OR  I Ai 
PROliU  l.<  SOL  VINE, 

PRCC:;)!JRAL  EVi  :;T5 
PRODUCTiUN  SYSTEMS 
PBOOUCTIVITV  IfCtd.OiOGV 
PRGGRAEV  VEkIEICAIION 
PSVOiOLOGY 

srcor.f.iTic*.  devices 

HI T’RI  SI  NilSG  RE  Al-WOk':  R uNLtWLCUGE  IN  RELATIONAL  PRODUCTION  SYSTEMS 
RE  .1  Cl. ij T ION  THiOREM  PROViEiG 
RiEiO'..1HCE  1 1 Kfi  Tc  D PWEXItSSi  S 
HI  1 HILVAt 

RODO'ICS  COOPERATION 


Bna  ACQUi'jiTiOS  CAPASkiTirS 
SCf.Nf,  SEtiMlNIATlON 

StMAuric  nris 

A 3t  MANTIC  IJ[  IWOBA 
SEMANTIC  I/ITWOWKS 
SENTENCE  ANli.G  IN  CONTEXT 
SENTENCE  MEANING  IN  CONTEXT 
SEHlAl  PAT  :EWN  ACQUISITION 
SEVEHA'.,  GOAlS 
SHAPE  CWAMMftRS 
SHAPE  TOnCLOCV 
SlMDi  TANEOUS  ACT  IONS 
SNARING  OBAWTNS 
SOFTWARE  INTERRUPTS 
SPEECH  UNDERSTANEiIN'G 
STATE  DESCRIPTION  MODELS 
STOCHASTIC  MODELING 
STORAGE  RCDLXITION 

strlictured  pattern  recognition 

SVMROi  M iPPlNG  IN  BASEBALL 
SVI'XTHROK.ZATION  Of  CONCURRENT  PROCESSES 
SVNTACT’C  METHODS 
SVNTAK 

SYNTHESIS  OF  line  DRAWINGS 

TELEOlOGICAL  Pi  ASO.'JiNG 

TtMl’OCAl  SCENE  ANALVSiS 

TEXTURE  ANALYSIS 

THE  AKVICLI  BY  ALLEN  NEWELL 

THi  RAY  ARE  A CIRC!  E 

THE  REWKI  El  Y OEHAIf 

THE  COMPlJIfWS  AND  TK'UGhT  AVMHO 

THE  DAT  IS  or  The  WORlf)  COMPUTES  CHESS  CONfEHENCt 

THE  OiDlX:'.  IVf  PAIHEINDES 

THE.  OR!  VtUS  OiRAit 

THE  ENVIPONMI  If  I 

THE  ECDERAl  judicial  system 

THE  game  Of  POKf  R 

the  history  or  ai 

THE  HUNGRY  MONNiV 

The  INSANE  HEUPj'TIC 

The  language  PASCAL 

THE.  EOC/iTIONOf  OiJJICTS  IN  MAGAZINES 

THE  LOGICAL  RLiEUCTlON  Of  LISP  DATA  BASES 

THE  MfTA-SVMiiOLiC  SIMULATION  OE  MULT iPBOCESS  SOFTWARE 

THL  METAmAThLS^AIICS  of  MJSPOR  M11SP2 

the  nomination  of  lYOMlNlLS  BV  A NATIONAL  NOMINATING  COMMITTEE 

the  CSTOGLNY  or  NON-INOEPENOEUT  SUBPROBiCMS 

THE  PARRY  SIMULATION  OF  PARANOIA 

THE  PERFCRMANCC  OF  PATTERN  NLATCHlNG  RULES 

THE  SIX  SEVEN  [ iCliT  NINE  GAME 

THE  STOCK  W/iPt;ET 

THE  STRUCTUTC  CF  ANY  VARIETY  OF  COMPUTER  TERMINAL 
the  TECH-li  CHESS  PfiCCHAM. 

The  WLAK  lOC'C  OF  PROGRAMS 
THREE  DIMENSIONAL  NLODELS 
time  COMP!  I XITV 
TIME  OP  SPACE  BOUNDS 


AIXIS 


UNf)tR!^TAM)IK‘G 
UNiroPM  PRCX3I'  paoCfouHf.; 

USiNC  S-l-CHAPiiS 
ViaUAL  COMMilNlCATiONi 

VISUAL  PLANtS  I'.  THE  BECOCKiliOr*  OE  POLYHEDfiA 
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Appendix  ni-C-3.  Al  Retrieval  Language  Grammar:  AIX05 


<SFNT>  . [ ] 

<SS>  - ANY  AOSlPACT'i  HCrt  IfiJlh'G  TO  tJTOPICS. 

AKK  »SAUTtiOR/S>  CITED  BV  ANY  Of  THOSE 
ARE  <JAUTHOH/S'  CITED  IN  ANY  RECENT  PAPERS 
ARE  cSTOPlCb-  DISCUSSED  IN  RECENT  JOUBNAES 
ARE  <t(OPlCS>  N.I  NTIOnCO  ANYWHERE 
ARi  .$T0P1CS>  MENTIONED  IN  AN  ABSTRACT 
ARE  ANY  ARTlCvi  S ABOUT  <ST0P:CS> 

ARE  ANY  ARTICLES  BV  ^SAUThO»/Ss. 

ARE  ANY  BY  -.SAUTHOR/S> 

ARE  ANY  NEW  HOOKS  BV  <SALlTHOR/S> 

ARE  ANV  or  THE  PAPERS  ON  tSTOPlCS>  ALSO  ABOUT  <JT0P1CS> 

ARE.  ANV  or  THESE  BV  <SAUTilQR/S> 

ARE  ANY  OF  THESE  FROM  AN  ACM  SESSION 

ARE  ANY  OF  THESE  FROM  THE  K IP  SESSIONS  IN  THE  MONTH  OF  JUNE 
ARE  ANY  PAPERS  ABOUT  <ST0P1CS> 

ARE  ANY  RECENT  ISSUES  ABOUT  <ST0P1CS>  BUT  NOT  <STOPICS> 

ARE  NOT  SOME  Of  THESE  FROM  COMPUTING  SURVEYS 
ARE  iHf  PE  ANV  abstracts  V/hICH  RCEfR  TO  <STOPICS> 

ARE  O'Eni  ANV  abstracts  V/(ilCH  RCrril  TO  PAPERS  BV  <:JAUTH0R/S> 
are:  ThEPE  ANV  ARTICI  ES  ABOUT  <STOPICS> 

ARE  THfnr.  ANV  ISSUES  ABOUT  <SfOPICS> 

ARE  THE  PE  ANV  N'EY/  ISSUES  CONCERNING  <S'OPlCS> 

ARC  ^^I^I!C  ANV  N'EY/  PAI-I  RS  ON  .ITOPICS.-. 

ARC  IHERI  ANV  PAPERS  Ir.AT  MENTION  <EIOPlCS> 

ARC  IHCRC  any  RI.CEWT  ARliC.I  S IN  CACM 
ARC  IHERI,  AI.V  Rl  CENT  POOKS  ABOUT  <$T0?1CS> 

ARC  CirWi'  SOMi  PAPl  RS  ON  <tTOPlCS> 

ARi:  YOU  AlV/AVS  THIS  SI-OW 
ARC  YOll  Rl  IIIJIAR:  V THIS  SLOW 
ARC  YOU  USllAiLV  SO  S'.  OW 

AR(  N T (III  PC  AN--  AUSTRACl'o  SINCE  NINETEEN  SEVENiV  FIVE 
CAN  ! riAVI  IHISI  AUSTRACU S uSTED 
CAN  YOU  IN  IP  Ml 
CEASE  PRINTING 

CHOOSE  AMONG  VOi  i(M(  S BEKIRC  NINETEEN  SIXTY 

COULD  YOU  Ri  '.RitVl  SOMt  VhlNC.  FROM  <$1NF0RMAT JON*AND.CONTfiOL>  DISCUSSING  <tTOPICS> 
DID  .-EAUTHOR/Sj  PRlT.I  1. 1 A PAPER  AE  IjCAl 

DID  -{AUT!I0R,'S>  PRIM  ',1  A PAPi  R AI  THE  IFiP  MEETINGS  IN  SEPTEMBER 
DID  ^SALiT.IOS/S;.  PRLSII.T  PAPERS  AT  lElP 
DID  c$AUT;,DR/S-  present  Pf.PcRS  AT  IJCAI 
DIO  rtAUTnOR.'S  ■ PuBLIS.i  A PAPIR 

CIO  lSauthor's--  write  a hook 

DID  tSAUTf  OR/S:-  WRITE  A UCOK  RECENTLY 

DIO  r<;AUr.iOR/S-  Y^RIIE  A PA=i  H ImIS  Yt  \P 

DID  ANY  - I.AI-JCA-RAAI  > PAP(  RS  CITE  <SAiJFriOR/S> 

DID  AfCY'  (*'C.  I'A{.IkSC1Il  ..SAu’i '■'CR/S' 

DID  ANY  IfEL  C0:.vf  NT1C.YS  PUD'-ISm  PROCElOINGS 
DID  ANY  OF  THOSE  PAPERS  CITE  <t;AUTHOS/S^ 

DID  ANVCNi  publish  ABOUT  ^STOPICS'  IN  COMMUNICATIONS  OF  THE  ACM 
DIO  the  SIGARf  newsletter  PuHLISh  ANYTHING  IN  OCTOBER  OR  NOVEMBER 
DIDN'T  THAI  PAPL  R OUOTE  .S/CUnOR/S> 

DO  A.,  OULiil'S  T/,KE  THIS  LONG 

DO  any  ARTIClISON  <$:0PICS'  in  addition  CONSIDER  «STOPICS> 

DO  ANV  AKTiClIS  ON  <STOP;CS'  MtNTiON  *JTOPICS> 
ljO  any  E.RTIClES  refer  to  ■<5T0P1CS> 


no  A'vv  Mi'HOpf.  ni sch');;!-.  -f 

00  AfxiY  Y;.W  A»T;cil‘,  K(KHilO\  rJTOPICS' 

00  AI^V  ()»  7)1!  A«SlflA(:!S  »J(I  !i710Ai  iS70PiCS> 

00  A!VV  Of  ’MISI  AIOO  D!SC(JSr,  -iTOPICSi 
DO  A';V  ()(  THi  Si  Al‘.0  Mf  li71C.\  <ST0P1CS> 

DO  ANV  Of  f‘ilSi  ClTt  <JAu!i.!!S/S> 

DO  ANV  01  fHfSI  t.<KMT10t)!  <?rOPICS> 

DO  ANV  01  TIiOSr  PAf'f  HS  ‘.'rNi  IC.N  <JTOPIC:S> 

DO  ANY  PAPi  WI,  AiiO.ii  t$TOPiCS.  AlSO  COTBIDEB  <$TOPrCS> 

00  ANY  PAPi  HS  C'lf  iSA‘JTfiOP/Si> 

DC  ANY  PAP;:<;;  r i^cor.s  .^TCPifs-. 

DO  ANY  PAP!  PS  DISCUSS  ^iTOPICS>  GUT  NOT  <JTOPICS> 

DO  ANY  PAPlifS  ON  .iTOPK:S>  fXlST 
DO  ANY  pap;:-!.  !)iis  YI  AN  Clff.  <SAUTHOS/S> 

DO  ANY  PfC.'M  ACM  CONFLP!  f.CiS  CONSIDER  <tTOPtCS> 

CO  ANY  WKCrNT  flOOKS  CITE  ..SAuTflOP/S> 

DO  ANY  Pf  C.-  NT  nCOl.S  N(f  NTI077  <STOPICS> 

DO  ANY  KffKNT  jOiTPNAl'J  DISCUSS  .STOPICS-. 

00  AljY  to  CENT  SUMMAPllS  DISCUSS  <$TOPICS> 

DO  MANY  AliSTPAClS  OiSC'jSS  <C.AUTl.OR/S> 

DO  MANY  AliSrtOACUS  LUSCUSS  -^lOPlCS.i 

DO  filS('ON;.IS  KS'LN  COMi  fAS-fP 

DO  TUI  V WGPr.  A1  THE  CM  NESi  ARC),  LABS 

DO  YOU  HAPPEN-  '.0  iiAVE  ANY  f.'t  P.f  NT  PAPERS  ON  <STOPICS> 

DO  YOU  have  AI.V  AHllClES  UN  »STOPiCS' 

DO  YOU  HAS  I ANY  NEW  PAPi  i.S  ON  <tTOPlf:S> 

CO  YOU  (lAVI  ANY  N'tC.ENT  PAf'ERS  ON  <STOPICS> 

DO  YOU  HAS'l  ANY  SUMMARIES  ABOUT  <STOPICS> 

DO  YOU  HAS'E  NEW  PAPE  lES  ON  ^tTOPlCS^ 

DOES  <ST0P1CSh  get  niSCo'SEi)  ANYWHERE 
DO'S  <5T0P1CS  • Gt  T ME  NTiONtn  ANYWHERE 
DOES  hE  VYORK  at  CMU 

DOES  IT  AlV/AVS  take  (HiS  lONu  TO  ANSWER  ME 

DOES  SHE  V/OSi;  AT  ThE  if.STiTU’t  EOR  SEMANTIC  AND  COGNITIVE  STUDIES 
DOES  THAT  Y.RTICa  MINilON  <ST0PIC5> 

DOE !;N  T This  PAPi  R PET  I III  N'CE  .•■N  lEf'l  TRANSACTION 
OON'l  GET  Ml  ANY  ARllCIES  SY-.CH  MENTION  <tTOPlCS> 

DURING  WH'AI  MU'.THS  W(  PE  li.tV  PL.3U3HED 

EINiSH  I PINT  IN.';- 

CIN'ERAU  A COPY  or  fliOSE 

Gt'l  ME  ANN’  (lOfTS-;  Wn;  I T' f-  BY  ^$AuTr:OR/S> 

G'i  ME  rS'l  RVl'P-M.  ON  .S'OPIC''. 

GIVE  Mr  ANV  ABSTRYiC'.S  MEMinNlNG  iSTOPICS>  BUT  NOT  <JT0P1CS> 
GIVE  Mr  ANV  AR.'iGEI  S AGOul  -STOPiCS' 

GIVE  ME  any  PAr'-ERS  ON  <JT0P:CS>  fROM  jUNE  TIL.,  AUGUST 

give  ME  CNi.  P,.IASL 

GIVE  Mi  SOMI ’rSNG  MrNlK.'MNS  -STOPICS> 

GIVE  ME  the  DA'!  or  THAI  AiilDRACT 

GIVE  't'Eir  AU'ir^OR  AM)  OAi'  0»  'LACEi 

t-IAS  1 $ A UTrrOR/S  > ETEi  N RI  ■■  I Ni  '.Ci  I)  A A'N\'  OE  T HOSE 

HAS  ^iAUTHOR/S>  PUGr.iSHt.)  A!,Y  PAPERS  THIS  YEAR 

HAS  i$AUTE'OR/S'  PUGLlSHfi)  ANYTHING  RECENTLY 

HASN  T ..SrOPICS-  HEIN  GONSiOERi  D IN  COMPUTING  REVIEWS 

HASN'T  A CURRi  .'A  REi  CRT  ON  tSTOPICSi  BEEN  HELI  ASED 

HAE^E  < $ A'l) T HCR/S  ' Pl! Gl  I 'Srr r ,)  rrijS  YtAR 

HAVE  AI.V  ARTiC.i  S A' Pi  .VRt  1)  V,'H:Ch  MENTION  <STOPICS> 

HAVE  ANV  NEW  PAPi  HS  BY  -$AU7riOR/S>  APPEASED 
HAVEN'l  YOU  r IMSHLO 
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HElI' 

HOW  BIG  IS  The  data  base 

HOW  CAfJ  I USE  TriE  SV3TEM  EFflClEHTLY 

HOW  lOHG  DOES  IT  TAKE 

HOW  MAU'V  ABSTI.'AC:TS  AHf  THERE  OH  <JTOF'iCS> 

HOW  MANY  ABSTRACTS  HCFER  TO  <STOPlCS.-' 

HOW  MANY  ARTICLES  DISCUSS  <STOPICS> 

HOW  MANY  ARTICLES  ON  <STOPICS.-  ARE  THEBE 

HOW  MANY  ARTICLES  WERE  WRITTEN  BY  <JAUTHOR/S>  AND  NOT  <SAUTHOR/S> 
HOW  MANY  BOOKS  DISCUSS  <ST0P1C3> 

HOW  MANY  ROOKS  Wl  HE  E-RDDUCEO  FROM  MARCH  TO  DECEMBER 
HOW  MANY  BOOKS  WEHE  WHIITEN  BY  <JAUTHOH/S> 

HOW  MANY  OF  THFSE  AISO  DISCUSS  cSTOPICS> 

HOW  MANY  PAPERS  AH(  AtlOUi  <STOPICS> 

HOW  MANY  PAPfHS  CONSIDER  <iTOP!CS>  SIMULTANEOUSLY 
MOW  MANY  PAPERS  DISCUSS  vSTOPICS> 

HOW  MANY  PAPERS  FROM  APRIL  THROUGH  AUGUST  CONCERNED  <$TOPICS> 
HOW  MANY  PAPERS  HAVL  <SAUTHOR/S>  WHITTEN  SINCE  JANUARY 
HOW  MAITY  PAPERS  RCfl  H 10  <ST0P1CS> 

HOW  MANY  PAPERS  TEIIS  YFAO  DISCUSS  <STOPICS> 

EIOW  MANY  PAPERS  WERE  WHIITEN  BY  <SAUTHOR/S> 

HOW  MANY  RECENT  ISSUES  CONCERN  eJTOPICS> 

HOW  MANY  REEEIII:N'CES  ARE  GIVEN 
HOW  MANY  SUMMARIES  DISCUSS  <STOPICS> 

I AM  interested  in  cST0P1CS> 

I AM  ONI  V INTLO' SUO  IN  PAPERS  ON  <tTOPICS> 

1 DEMAND  AN()Tnr«  ARTICIE  AFTER  AUGUST  NINETEEN  THIRTEEN 
rO  LIKE  TO  KNOW  THE  PUBLISHEHS  OF  THAT  STORY 
I'D  LIKE  TO  SEE  THE  MENUS 

15  <SAUTH0R/S.-  but  NOT  <$AoT;iOR/S>  cited  in  SOME  OF  THOSE  ARTICLES 

IS  <sauthor/s>  cited  by  those  abstracts 
IS  ^SAUTH0R/S>  cited  in  ANY  or  these 
is  <stopics  - discussed  anywhere 

IS  <$T0PICS'  DISCUSSED  IN  A RECENT  SUMMARY 

IS  <$TOPICS>  MENTIONED 

IS  <STOPICS>  MfNiIONEO  ANYVYHERE 

1?  <STOPICSs  MtNllO.VCO  INANABSIRACT 

IS  <tTOPICS  . RFK  RRFD  TO 

IS  <ST0P1CS>  RHKfRFD  TO  ANYWHERE 

IS  THAT  ABOUT  -STOPlCS' 

IS  THERE  A RECENT  ARTICLE  ABOUT  <STOPICS> 

IS  THERE  A RECENT  PAPER  AitOlIT  <:$T0P1CS> 

IS  there  a RECENT  PAPER  MENTIONING  <$TOP:CS> 

IS  THERE  AN  ARIICII  ABOUT  <;TOPICS> 

IS  THERE  AN  t(  IP  CONVFNflON  ISSUE  FROM  MAY  OR  JUNE 
IS  THERE  ANY  I HINT.  NEW  REGAKDINC.  <$TOPiCS> 

ISN'T  iSrOPlCS>  MINTIONFD  IN  AN  AH'STHACI 
ISN'T  fHEliE  AN  AHTICd  ABOUI  <ST0P1CS> 

KILL  the  LISTING 

LIE  ME  LIMIT  MVSCl  I TO  Hil'ORTS  ISSUED  SINCE  NINETEEN  FIFTEEN 
LET  US  CONEINE  OURSEl  VIS  10  JOURNALS  AFTER  FEBRUAHV  NINETEEN  FIFTY 
LI  VS  RESTRICT  OUR  ATTENUO.V  TO  PAPERS  SINCE  NINETEEN  SEVENTY  FOUR 
LIST  BE!  WI  ( N TWl  ;VI  AND  TWENTY  OF  THEM 
LIST  the  AB'IVHACVS  BY  <SAuTr03/S> 

LIST  TNI  NtXT  fOURULN  MuNDfiCO 
NO  MOPE  PlLAGC 
NO  Thanks 
OK 
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PLl.AGC  Hli  ('  MC 
P:  LAGC  UST  Th!"  AUTHORS 
FLLAGC  WAKt  MK  A TllC  Of  THOSE 
PlLAGE  TE«W1N7,:c  TKAfiSMli’TIUG 
PRINT  THE  NEXT  ONE 

PRODUCE  A COPV  Of  THE  NEWEST  EIGHTY  AHTiCLES 

QUIT  LlSTII.G  PlI  AGE 

SEiECT  IPOM  ARTICLES  ON  ct:OPiCS> 

SHOW  ME  ITS  PUHLISHER 
SHOW  ME  THE  LATEST  ElEVEN 
STOP  TRAI.SM:  IT  INC  Pi  EASE 
SUBSELEC;  PROM  vSiCPICS> 

SURE  THARXS 

TELL  ME  the  TITlES  OE  THE  EARLIEST  TEN 
TELI  ME  WHAT  TO  DO 
THANX  YOU  EM  DONE 

THE  ARfA  I Al.<  ■RTERI  SIED  IN  IS  <;TOPiCS> 

THE  ARI  A I'M  IMERESTEi)  IN  IS  <JTOPICS.> 

THE  EIRST  TWO 

THE  i ATEST  SIXTEiN  PELAGE 

TKANSMIl  THI  NfXT  EiCHTELN 

TRY  TO  GEl  GiiRVtYS  PRINTED  IN  TmI  lAGT  EIGHTY  MONTHS 

WAS  <SAuThOB|/S>  CITCi)  BY  ARV  REPORTS  ISSUED  IN  THE  LAST  NINETY  YEARS 

WAS  <saiithor/S'  cited  in  Thai  summary 

WAS  <STOPiCS-  MiNTiONfO  SOMElVHf  ME  IN  RECENT  TIMES 

WAS  <ST0(’K;EW  WWiTTLN  LM  RECENTLY 

WAS  n PlJUi  lG'li  0 iiV  <STIir.  A;.SOCIAT10N.EOR.COMPUTAT10NAL.LINGUISTlCS> 

WAS  IT  Pliiii.;GHID  HV  ThI  .10  ..RNAl  Of  THE  ACM 

WAS  THI  «l  A COl.M  REKCl  IN  TML  USSR 

WASN'T  vtlOI’lCSv  MIMiON'Er-  RECENTLY 

WASN'l  v$IOi>iCS>  RinilRED  10  SOMEWHERE 

WL  DESIRE  A PROCt  I DING  0*'  Trr  ACM  MEETING  REE  EHENCEl)  BY  ctAUTHOR/S> 
W[  WANT  SOME  RLViCV/S  CONCERNING  <STOPICS> 

WE  WISH  TO  GlV  THE  latest  EORTY  ARTICLES  ON  <$TOPiCS> 

WL'D  LIKE  10  S(  i In;:  TiTlES  EROM  PROCEEDINGS  OP  THE  ACM  CONPERENCE 
WLTII  IN'TI  RESTED  IN  .JTOPICS' 

WfRI  INLLRiSTED  IN  ARDCLiS  E'DilLiSHEO  IN  THE  LAST  THIRTY  YEARS 
W(  VE  HECK  INTERt  STEi)  IN  --STOPICS' 

WERE  ANA  OE  TMi  SI  API  iCL  ES  WRI  (Tt  N BY  <JAUTfiOH/S> 

WEi<i  ANA  OT  TH,  ‘ i P.jitilSHf  D IN  ThE  SUNStilNE  STATE  OH  IN  THE  US 
WERE  Al'Y  OP  Thc-I  'A'Plllf. N by  «SAUThOR/S> 

WENT  ANY  PUtM  IjmED  Ai  TER  .;.M  NlNErEEN  SIXTY  PiVE 
WE'D  THri’i  ANY  ARIlCll  S AHiniT  <$TOP;C‘:> 

'WEREN'T  GDMi  A'-',  .CilS  PuUi  ISHEO  ON  <STOPiCS> 

V/HAI  AI'OUT  <LAUiHOR/S> 

WHAT  ALsoui  • sroPics-. 

WHAT  ADOK."-';  IN  GIVTN’  fOR  ImE  AUTHORS 
WHAT  ADDPfSGI  !'  API  C.lS'EN  TOR  ThE  AuTHORS 
WHAT  ANT  50MI  OF  THL  AREAS  OP  <tTOPICS' 

VYHAT  /iRE  T.-iE  KE  Y PHRASES 

WHAT  ARE  IhE  TJIiES  Of  THE  RECENT  ARPA  SUR NOTES 
WHP^T  PJiP  I Hi  IR  A*”  F II  1 AT  iON5 
WHAT  nCOi-S  l.'PNTION  <$T0P:CS' 

WHAT  CAN'  ! DO  10  SPIEO  YOG  lIP 

WHAT  CAT,'  ThF  SYSTEM  DO 

WHAT  CONPf  Rl  NCf  WAS  A"  R.JTOERR  OR  AT  SSI 

WHAT  CONPI  RIT.CE  WAS  AT  WATSON  RESEARCH  OR  AT  ILLINOIS 

WHAT  no  i HAVE  TO  DO 
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WHAT  TACl'j  Atd  STOBf  0 

WHAT  HAS  <$AIJThOR/S>  WB!tTfMAT£LV 

WHAT  HAS  vSAUTHOR/S>  WfilTTfN  B£C£NTIV 

WHAT  HAVC  »tAL)THOR/S>  WRITTEN  LATCLV 

WHAT  15  HTH  A1  ULIATION 

WHAT  IS  HIS  ArnUATION 

WHAT  IS  KNOWN  AHOUT  EVtWY  ARTICLE 

WHAT  IS  Tui  SII^E  or  THE  DATA  BANK 

WHAT  IS  the  title  or  that  PAPEH 

WHAT  IS  THE  TlTlL  OF  THE  EARUtST  ONE 

WHAT  IS  THE  TITLE  OF  THE  MOST  RECENT  ONE 

WHAT  ISSUES  DURING  JANUARY  AND  JULY  CONCERN  <JTOPICS> 

WHAT  KEY  WORD  BEl  ATCG  TO  <ST0P1CS> 

WHAT  KEY  WORDS  SHOULD  1 USE  FOR  <$TOPICS> 

WHAT  KIND  OE  MENUS  ABE  THEBE 
WHAT  KINDS  Of  SUBJECTS  ARE  STORED 
WHAT  MUST  I ASK 

WHAT  PAPERS  ON  c5TOPlCS>  ARE  THERE 

WHAT  SHOLIlO  1 ASK 

WHAT  SHOLIlO  1 EAV 

WHAT  SORT  or  SUMMARY  IS  AVAILABLE 

WHAT  SORTS  OE  <$  rOPir.S>  ARE  WREiTEN  UP 

WHAT  SUBJECT  CAN  I REOUEST 

WHAT  TOPIC  MENij  CAN  1 CHOOSE 

WHAT  TOPICS  ARE  RELATED  TO  <STOPICS> 

WHAT  TYPES  or  <5RrTRlEVAL.CAN.HEARSAV>  DO 

WHAT  WAS  ITS  TITLE 

WHAT'S  the  PUBLISHER  Of  THAT  PIECE 

WHEN  WAS  <;SHIJMAN.?R08cLM.S0LViNC>  WRITTEN 

WHIN  WAS  <STOriCS'  LAST  MENTIONED 

WHEN  WAS  <STOr!CS>  LAST  RCEfURtO  TO 

WHEN  WAS  IT  PU8E1SHE0 

WHEN  WAS  that  ROOK  WRITTEN 

WHEN  WAS  That  paper  PUBLiShEO 

WHEN  WAS  the  last  paper  BY  <SAUTHOR/S>  PUBLISHED 

WHEN  were  <$TOriCS>  LAST  SETERREO  EO 

WHEN  WILL  YOU  HAVE  THE  ANSVYEH 

WHEHi;  ARE  <STOniCS>  REFERRED  TO 

WHERE  DID  That  article  APPEAR 

WHERE  DO  THEY  WOPK 

WHERE  DOES  HE  WORK 

WHERE  IS  •.ironies^  M'NfioNEn 

WHICH  <$Al.UXf>  CONTAINED  <tTOPICS> 

WHICH  ^?COGNlTlVE.PSvC-iOLOCV>  CONTAINED  tJTOPICS> 

WHICEI  <S(:OGMr!VF. PSYCHOLOGY^  CONTAINS  <tTOP:CS> 

WHiCEl  AHSTWACIS  CONCLRN  <£TOPICS> 

WHICH  AHSIRACIS  REFI  R TO  .-STOPICS> 

WHICH  ARl  ICl  ts  CONCERN  .,f,TOPiCS- 

WHICH  articles  havl  concerned  <$T0PICS> 
which  ARl  let  ( S ON  <;T0P!C.S-.  Also  CONCERN  «;tTOPICS> 

WHiCEl  ARl  K:l;  S RLF I R 10  these 

WHICEI  AUTHORS  WORK  AT  IIAMHURC  OR  AT  EDINBURGH 
WHICEI  authors  work  AT  NIH  OR  AT  STANfORD 
WHICEI  AUTHORS  WORK  WITH  SU'ME  X OR  AT  SUSSEX 
WIHCEI  COCKS  ON  ^JT0PICS>  WERE  PUBLISHEO  RECENTLY 
WHICH  BOOKS  WIRE  WRIITEN  BY  <SAUTUOR/S>  SINCE  LAST  YEAR 
WHiCEl  computing  SURVEY  ARTICLES  REEATC  TO  <STOPiCS> 

WHICH  computing  SURVEYS  CONTAINED  THE  ARTICLE  BY  <tAUTHOR/S> 
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WlilCH  CONfl.HiliCIS  WfHI.  Al  MASSACtIUSCTTS  09  AT  ROCHCSTEH 
wiiiCH  IS  thi:  olocst 

WHICH  NOTtS  CM  <ST0PICS>  AISO  DISCUSS  <ST0P[CS> 

WHICH  Of  fHl  M ITlSCUSSrS  ^3IOPiCS> 

WHICH  Of  (Ml  Si:  AHPl  ARCO  kCdUNTLY  IN  THE  IEEE  TkANSACTlONiS 
WHICH  or  (hlsi  ahi;  hy  <sAuTfiOR/s> 

WHICH  or  THESE  CITES  <sauthor/s> 

WHICH  Of  THESE  WAS  WHITTEN  DV  <tAUTH0R/S> 

WHICH  or  these  were:  whiiten  by  <tAurfK)R/s> 

WHICH  ONES 

WlilCH  PAPIH  mentions  <stopics> 

WiUCH  PAPERS  AITE  OH  <STOP!CS> 

WHICH  PAPERS  SV  ctAUTflOfi/S>  ARE  REFERENCED 
WHICH  PAPERS  CITE  <JAUTH0R/S> 

WHICH  PAPERS  DISCUSS  <STOPICS- 
WHICH  PAPfRS  HAVE  MENTIONED  <$TOPlCS> 

WHICH  PAPI  WS  ON  <JT0P1CS>  Ai SO  CONCERN  <tT0PlCS> 

WHICH  PAPf  RS  ON  iSTOPICS  - Al  SO  OiSCUSB  -;JT0P1CS> 

WHICH  PAPi:RS  ON  <JTOPICS>  ARE  ABOUT  <JT0P!CS> 

WHICH  PAPERS  W(  IP  WlTlilEN  AT  NRE  OR  AT  SMC 
WHICH  PAPERS  W(  i<E  WRifItN  BY  <tAUTflOfi/S> 

WHICH  RECENT  JOuRNAlS  REEfW  10  <ST0PICS' 

WHICH  SORT  Of  <SRFTRI£VAl.<cYS>  CAN  I SEEK 

WHICH  STORIES  IN  The  SIGART  NEV/SEETTER  have  been  DISCUSSING  <JT0PICS> 
WHICH  SUMMARIES  ON  <iTOPICS>  CONSIDER  <JT0PICS>  IN  ADDITION 
WHICH  TECHNK.AI  PAPERS  WIRE  WRITTEN  BV  <iAUTH0R/S> 

WHICH  TlTlES  CONTAIN  THE  PHRASE  <JT0PICS> 

WHICH  WAS  THE  LAST  ARTlCEf  BY  iSAUTEK)R/S> 

WHO 

WHO  MAS  WRITTEN  ABOUT  cSIOPlCS> 

WHO  WAS  QUOTED  IN  THAT  ARTICLE 
WHO  WAS  rill  AUTHOR 
WHO  WLRi  IfiE  AUTHORS  OE  THAT  BOOK 
WHO  WROTE  IT 

WliO  WROTE  PAPi  RS  ON  <$TOriCS>  THIS  YEAR 
WHY  IS  Till  SVSTi  M SO  SiOW 
V.'CjULD  YOU  list  up  0 GEVI  NIEEN 
WRITE  A f ILL  Of  THOSE 
YES  PLIIASC 


121 


AIK05 


<S;1NF0RWATI0N.AND.C0NTR0L>  • IKTORMATION  ANO  CONTROL 
<scor.u'mvf:.PSvcHOLOGy>  - cocNnivt  psvcholocv 

<5THF..WORLD.COMl'iJT£R.CH£SS.CONFiHENCE>:’.  THE  WORLD  COMPUTER  CHESS  CONFERENCE 
<SAi.JOURIJAi',  . AI  JOURNAL 
<SA1.T£XT>  . Al  TEXT 

<STHF.Ar>nCClATION4FOR-COMPUTATIONAL.lINGUISIlCS>  ^ THE  ASSOCIATION  FOR  COMPUTATIONAL  LINGUISTICS 
<SRrTRILVAl  .CAN.UEARSAV*  ^ RCTHlEVAl  CAN  HEARSAY 
<SHIJMAN.PROBLEM.SUi  V)NG>  . HUMAN  PROBLEM  SOLVING 
<$RrTRIEVAL.Ii:EVS»  . RETRIEVAL  KEYS 


122 


AIKOS 


<5AUTHOR/S>  . <AUTHOR> 

<AUTHOR>  <ANDOR  l>  <AUTHOR> 

<AN'DiOR  1>  - AND 
OR 

<AUTHOR>  - AI.Lf  I.  COLLINS 
ALLtH  NEWELL 
ANN  PJBIN 
ANT'tONY  MARTELLl 
A7RIEL  HOSENEELI) 

BERNARD  Mt  LTZER 
BERT  RAPHAEL 
SILL  WOODS 
BONNIE  NASM-WE88ER 
BROCE  BUCHANAN 
carl  HEWITT 
CHRISTOPHER  R1ES8ECK 
CHUCK  RIEOER 
OANNV  BORROW 
DAVE  RUMH  HART 
DAVID  MARS 
DAVID  MICHIE 
DICK  SEITZER 
OONALO  NORMAN 
C-OUG  Li  NAT 
DREW  Mi'-OERMOTT 
OREVEUU 
EARL  HUNT 
EARl.  SACEHOOVl 
EO  rr.iGCNHAUM 
£0  HISEMAN 
ELI  lOT  r.OLOwAY 
ERIK  SANDLWALl 
EUGENE  CHARNIAK 
PnGENHAl/M 
FELDMAN 
GARY  HENDRIX 
GEORGE  ERNST 
GIPS 

HAN'S  BEHi  IKi  H 
HARRY  HARROW 
HERD  SIMON 
HLHI’ERT  CLOCK 
HILARY  PUTNAM 
HOLLAND 
HUGH  NAGEL 
IRV  SORE I 
ISSAC  ASIMOV 
JACK  MiNKEH 
JACK  MOSTOW 
JAMES  SLAGLE 
JEAN  SAMMET 
JFFEREY  IJLIMAN 
Jf  RDY  FEi  OMAN 
JOHN  GASCHNIG 
JOHN  HOLLAND 
JOHN  MCCARTHY 
JOHN  NE'w'CO'vitS 


123 


jOStPII  W;  i/'tWaAUM 
J'.lDfA  PlARi 
KLARi  PIN'GU. 

KLlTh  FRICf 
KtN  COLOV 
KIN  RALSTON 
KING  SUNG  ru 
LAURrUT  SIKLOSSV 
LET  ERMAN 
lEONAliU  IIHH 
LES  EARNEST 
LINDA  NIAS  i NT  I R 
MADELINE  GATES 
MARVIN  WINSKV 
MARY  NEWHORN 
MARY  SHA'.V 
MICHAEL  ARIlid 
MlKf  R/CHENEfi 
MINSKY 

MITCHEII  I.EWEV 
NEWEU 
NILS  NILSSON 
NILSSON 
NORI SU7UK1 
PAKUl A MCCOEDuCK 
PAT  WINSTON 
PERRY  LHOSNOVKE 
PETER  KuGEL 
RAJ  REODV 
RANAN  ElANLR.li 
RAYMOND  SPfiOULL 
PIOOV 
RICH  riKLS 
RICH  SMITH 
3ICEIAR0  MlCHALSKl 
RICHART)  V/A',0!NG£H 
RICK  HAVES  ROTH 

BOGLRT  pl:  IER 

ROGER  SCHANK 
RON  out  AICOCII 
R03ENE  ELI) 

SCOT  ’■  EAI*:  K<A.s 
SEVMObR  PAPER’ 
SIMON 

STEVi  COIES 
SIEVE  REEO 
STEVE  /lICKEW 
TED  SHORTllFEt 
TERRY  V/i!.'OGRAO 
THOMAS  MAR5iAi;0 
THOMAS  SVKLS 
UHR 

VIC  lEESE  ii 
VYAa  V REiOMHtftG 
WOODS 

WOODY  HTEE'SDE 
VORICK  WI.  I'.S 
ZOHAR  MAfVNA 
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<STOPICf;>  ■ <ANiliOR'TOPir.S> 

WIK'OCRAO  S ARTiCLE 
<ANOvORiTOriCS^  . <TOPIC> 

<TOPIC>  cAIJDOR  2>  <TOPIC> 

<A,S'l)'OR  2>  ’ and 
OR 

«rTOPiC'  - A ca:  monitor 

A COMMON  SENSE  ALCORITHM 
A GAI.<t-  model 
A LOSING  MOVE 
A MltlTlLtVtl  ORGAN)2ATION 

A PAC'.KET  BASED  APPROACH  TO  NETWORK  COMMUNICATION 
A PASTIA'  EVALUATOR 

A PROGRAM  SVNVHESI2EH  FOR  NETWORK  AROTOCOIS 
A PfiW.'RAMMlNG  APPRENTICE 

A PROOl  CHECKER  FOR  PROTOCOL  TERMINATION  EXPRESSIONS 
A RADIO  INTERVIEV/  ON  SCIEKCE  FioTlON 
A REGION  ANALV'ilS  SUBSYSTEM 
A STEREO  PAIR  OE  VIEWS 
A TASK  ORIENTED  DIALOGUE 
A THAUMATURGIST 

A THEOREM  PROVER  PLANNING  FOR  PROGRESS 
A TIME  DOMAIN  ANALV2ER 
A TUTOR  OR  UITORING  ON  TV 
A TV  RCPCRtER 
ARSTRACnCN 

ACQUISITION  OF  KNOWLEDGE 
ACTIVE  KNOWiLDGE 
ACYCLIC  ISOMERS 
ADAPTATION 

ADAPTIVE  PRODUCTION  SYSTEMS 
ADVISING  PHYSICIANS 
A! 

A1  LECTURES 
ALGEBRAIC  Rf  DUITlON 
Al  SOI 

AIGORITHMIC  AiSTHEI  ICS 

All-OR  HONE  SOLUTIONS 

AN  ADAPTIVE  NATURAL  LANGUAGE  SYSTEM 

AN  AS'U  MIII.V  ROBOT 

AN  AXIOMATIC  SYSTEM 

analogy  in  PROruLM  SOLVING 

analysis  OE  CONTEXT 

ANALYSIS  0(  SENTENCES 

AMTIEICIAI  INTLUlGENCE 

ASSIMILATION  Of  NEW  INFORMATION 

ASSCXTIATIVE  MEMORIES 

ASSOCIATIVE  MEMORY 

ALiGMi  NTED  TRANSITION  NETWORKS 

AUTOMATCO  DEDUCTION 

AUTOMATIC  CODING 

AUTOMATIC  COMPiJtATlON 

AUTOMATIC  MAIVTPA  GENERATION 

AUTOMATIC  PSOCREiM  SYNTHESIS  FROM  EXAMPLE  PROBLEMS 
AUTOMATIC  PROGRAM  WRITING 
AUTOMATIC  PROGRAMMING 
AUTOMATIC  PBOOE  OF  CORRECTNESS 

Automatic  thcorev.  psoving 
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AUTOMATION 

AXIOMATIC  SEMAMTICS 

AXIOMS  FOB  GO 

BACKGAMMON 

BELIEF  SYSTEMS 

BINDINGS 

BIOMEDICINE 

B9AIN  THtOBV 

BUSINESS  PROBLEM  SOLVING 

CARTOGRAPHY 

CASE  SYSTEMr. 

CAUSAL  REASONING 
CELl  ASaEUlILV  THEORY 
CHECKING  PROCFS 
CHESS 

CHESS  PLAYING  PROC'’AMS 
ClRCUr  ANALYSIS 
COGNITION 

COGNITIVE  ROBOTIC  SYSTEMS 
COGNITIVE  SCIENCE 
COMMON  SI  NSE 

COMMON  SI  USE  THI  OSV  FORMATION 
COMPl.l  K WAVEFORMS 
COMPUTATIONAL  LINGUISTICS 
COMPUTER  ART 

COMPUTER  BASEO  CONSULTANT 

COMUJIER  BASED  CONSULTATIONS 

COMPUTER  CONIROLLEO  MANIPULATORS 

COMPUTER  GRAPHICS 

COMPUTER  MUSIC 

COMPUTER  NETWORKS 

COMPUTER  VISION 

CONCEPTUAL  DESCRIPTIONS 

CONCEPTUAL  INFERENCE 

conceptual  OVERLAYS 

CONSIRAINT  SATlSfACTICN 

CONSTRUCTING  PROGRAMS  FROM  EXAMPLES 

CONSTRUCTION  OF  PROGRAMS 

CONTEXT 

CONTINUOUS  PROCESSES 
CONTROL 

COOr-EHAT  ;i.'G  SOURCES  OF  KNOWLEOCE 

CO^VIN'G  LIST  STRUCTURES 

CURVED  OH.il  r.TS 

CVBfRNETICS 

CYCLIC 

DATA  BASES 

DATA  BASES  FOk  INTERACTIVE  DESIGN 
DATA  STRUCTURES 
DECISION  theory 
DEDUCTION 

DEDUCTIVE  retrieval 
DENOTATIOSAL  SI  PIANTICS 
depth  perception 

DERIVATION  PLAN'S 
DESIGN 

ULSiL.N  automation 
DESIGN  IN  the  arts 


OCUCTICN  OP  lir.HT  SOUBCES 

DtSPLAV  T£«M!NAIS 

DflA(;ON 

DRIVING  A CAR 

OVNAMIC  BINDING 

DVNAMIC  CtUSURING 

DYNAMIC  PROGRAMMING 

ELtCIBONIC  CIRCUITS 

ELECTRONICS 

ENGLISH 

EVALUATION  FUTJCTIONS 
EXPERT  SYSTEMS 
EXPiANATIOJJ  capabilities 
FABLES  OP  FAIRY  TALCS 
FEATUPr  DRIVEN  SYSTEMS 

first  order  logic 

formal  SEMANIiCS 
FRAME  Theory 

framfs 

F RAMPS  a;;;)  the  e:nvir£>.nme:nt 

FU??V  KNOWLtllGE 

FU??Y  PKOnil  M SOLVING 

GAMI  OF  POF.f  H 

GAME  Pl.AVlNG 

GLNERAl  PlJPPOSE  MODELS 

GENERA 'ION  Of  NATURAL  LANGUAGE 

GLOME  TRiC.  MOOLLH'.'G 

GO  OR  GO-MOXU 

GOAL  SLI.EDNi;  COMPONENTS 

CHAIN  OF  COMPIj: AVION 

CHAEXMATICAI  INFERENCE 

GRAPH  INTERPHCIABlE  (GAMES 

GRAPH  MATCHING 

HI  ARSAV 

HLVEWOSTATIC  THEORY 
Ki.llHISTIC  PROGRAMMING 

heuristic  techniques 

HIlL  CLllHHlNC 

HUMAN  OCi’AViOR 

human  memory 

HUSTAN  VISION 

HYPOTHESIS  FORMATION 

IL'AGE  'NTcNSlIY  UNDERSTANDING 

IMPROVING  PfiCSHAI^S 

INDilCTJ'Yf  ASStliilOKS 

ISDUSTRlAl  APPi'-CAIION 

INEXACT  HCPPlSENTATION 

INF  ERE NC': 

IN'FERLaCLS 

lI.rER.T.'TIAL  OUiSTiO'N  A.NSWXRiNG 
INEORMAT.ON  processing  UNiVERSAlS 
IMIERITAHCE  of  PROPERTIES 
INTEILICENT  MACEiINES 
INTENTIONS 
IlvURACTiVE  DESIGN 
INTERACTIVE  KNOAVLEDCE  SYSTEMS 
INTERACTIVE  PROGRAM  SVNTEiESIS 
INTERPRETIVE  SEMANTICS 
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INVABIAN'Cf  FOP  PBOOIFM  SOLVING 
IS'VARIANCLS  IN  TUC  P£PC£PT10N  OF  FACfS 
INVLSYMtNf  ANALVGiS 
ITLHAUON 

KNOWLt  (ICC  8ASf  t)  SVSTEMi; 

KNOWK  OCt  SVSTEMS 
lAMIiOA  calculus 
LANGUAGE  COMPHCHENSION 
LANGUAGE  OtSIGN 
LANGUAGE  PAHAPHIIASE 
LANGUAGE  PASCAL 
LAN'GUAGE  PRIMITIVES 
LANGUAGE  UNDERSTANDING 
LARGE  DATA  BASES 
LEARNING 

LINEAR  LEXICOMETHV 

LOW  ORDERS  OE  RECOGNITION  PERFORMANCE 

MACHINE  INTEL!  IGENCE  IN  MEDICAL  OIACMOSIS 

MACRO  PROCESSING  FOB  AN  ON-LINE  MEWSUTTEB 

MANAGENUN'T  iN''ORT.<ATION  SVST£fc«S 

MI  AN3  res  COMPUTER  MOVIES 

MEDICAL  CONSULTATION 

MINIMAL  SPANNING  FORESTS  OR  TREES 

MOTION  IN  SCENE  OESCBIPTiON 

neural  NETWOPKS 

NEWSIEI  rtW  REPORTERS 

.VONOETERMIMSHC  PROGRAMMING 

OB.IICI  LOCATIONS  AND  MOVEMENTS  IN  NATUflAl  IMAGES 

OBJECT  WAMIPUATING  ROBOTS 

operational  reasoning 

OPTIMAL  PROBLEM  SOLVING  SEARCH 

OPTIMIZED  CODE  FOR  THE  TRANSFER  Of  COMMENTS 

PAPERS  BV  RILL  WCtOOS 

PARALin  ISM  IN  PROBLF.M  SOLVING 

PARTIAL  EVALUATOR 

PATTERN  OiRECirO  FUNCTION  INVOCATION 

PATTERN  MATCHING 

PATTERN  R(  COGNITION 

PCRCEPTRONS 

PhOTOCHAMMI IRV 

PICTuHr  kECOC.MTION 

Planner  I ii:c  languages 

PWEiIICAU  CAlCDLiTS 

PREI  I RI'UriAL  SEMANTICS 

PRK'I'S  TUKiRIA! 

PROEILI  M SOLVll  S 

pwocldural  evints 

PRODUCTION  SVSTEMS 
PROOUCTlVirv  TLCHNOLOCV 
program  VIHlIiCATiON 
PSVCHOLOGV 
RECOGNITION  DEVICES 

RFPRESCKTING  REAL-WORLD  KNOWLEDGE  IN  HflATiONAL  PBODKTION  SYSTEMS 
RESOLUTION  theorem  PROVING 
RESOURCE  LIMITED  PROCESSES 
RETRIEVAL 

ROBOTICS  COOPERATION 
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Bin  f ACO’JiSi ' 1'  W CAPABJlITltS 
SCi  .-.'F  '^f  WHJTAi  ION 
Sft.iANTic  nr  IS 
A Sf  Mf.MiC  nf.tWfiB!; 

SO.'AMIC  lir'W(iK'i,S 
StNItNCe  MI  ANINC  IN  CONTEXT 
sentence  mi  AWSu  IN  CONTEXT 
SCRIAI  PAITtk’N  ACQU'SITION 
SEVEHOL  r.OAlS 
SHAPE  crammai;'; 

SHAPE  (OrfllOGV 

SIMlii  lAUfOUS  ACTiONS 

SK'AKtNG  CkA'v'OItS 

SOETWABI  INrtk'«;JPIS 

SPEECH  iiNOERSTANOiNC 

state  iTESCRIP',  iON  MODELS 

STOCHASTIC  MOOCEI.Va 

STORACI  BtntX.TlON 

STBUCTllUrO  PATTERN  RECOi-NKlON 

SVI,<IIOl  I.IAl>P;;.'i.  IN  OAStItAU 

SVWTHHONIiATiON  0^  COTiC'JRMENT  PROCESSES 

SV'VTACIIC  Ml  T lilies 

SVM  AX 

SVN'HiSISOr  LiNt  DRAWINGS 

Til  iOiDL.iCAi  Ri  ASONIKG 

TiM'^ORAl  SCENE  ANALYSIS 

TtXTliWr  ARAL  vs ;s 

71, t ARl  ICtl  HV  AlLUi  NEWELL 

Tilt  PAV  A^'I  a (.iRL*ii 

T,,i  PERKEl;  V OlitAIt 

THE  COl.'PiJESS  AM)  TmOCl.  'T  AWARD 

THE  DAT(  S Oi  i'H:  WOBLH  COMPUTER  CHESS  CONE EREKCE 

The  DEOL'CTiVt  PATiulNOCB 

7 fiE  DEI  V' iiS  Ot  «A i t 

THE  ENVinCNMI  '. ' 

T'.I  FFCI.RA,  SYSTEM 

THE  GAM!  or  POALP 

Tile  HT5,OsV  \J.'  Al 

The  iiuNGE"'  MOAXlV 

THE  iNSAf.'I  hECRISTIC 

The  i iMvGLAGI  EASCAi. 

The  location  Of  OBjiCIS  IN  MAOAZiNES 

Tr,l  lOGICA;  Ri  .l-.iCTION  GE  lisp  data  bases 

Trii  Nif  iA  SVMHOl  !C  SITii.'i  ATiON  Of  MlJi  TiPSOCESS  SOETWASE 

Tr„  Vf!AMA’-.|M/iTK;S  or  Mi  ISP  OR  .VI  ISP? 

THi  .\0'.<i>.AT;0\  01  .VOMINi  tS  t.V  A NATIONAL  .NOMiNATING  COMMITTEE 

TilE  ONTO.SfNS' r.Ol.-iNOEP.  SDCNi  SUQPR08LLMS 

THE  PAP'WY  ATION  or  FAWANOIA 

Tf.i  pEhI T.HHiAi.cr  01  patter:,  matchi.nc  Rules 

The  Six  SLVfii  t icht  nine  gami 

ThI  STOCx  MAOx!  '. 

Tfi!  STRuCTil.'r  or  ANV  VARIETY  OF  COMPUTER  TERMINAL 

T.'n  ifCi*  li  ..i,i''':  Pro.. RA M 

Tf,!  Wl  Ai.  .OGiC  Of  PROGRAMS 

Ti  kI  1 .Oli.ii  I.SiCNAL  MO-OElS 

T.i.ii  C‘iM.':  1 *;■. V 

T it.Ti  [jli  SPmCI  HC.UT.GS 

TROiPiLi  Shooting 
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UfvOCWGTAS.iING 
UMTOh'M  PS(X>r  AROCLDURCS 
S- 1 -GWARhS 

ViSUA,  COMk<llli-;CAT;0\ 

Visual  PLA\f.S  IIJ  TKF.  RtCOCNUlON  of  POi'i'HtORA 


IV.  COLLECTED  PAPERS 


A i.c'ect.'on  of  papers,  ait  of  which  appeared 
Ifilern^jtiOnal  Co.-.ffcrence  on  Acoustics,  Speech, 
Connecticut,  Cay  9-11,  1977.  with  the  exception  of 
and  related  pruning  heuristic  for  combinatorial 
working  paper. 


m the  Proceedings  of  the  IEEE 
and  Signal  Processing,  Hartford, 
the  last  paper,  “A  halting  condition 
search",  which  is  an  unpublished 


A functional  description  of  the 

HEARSAY-11  SPEECH  UNDERSTANDING  SYSTEM 


Lee  0 ef'^tan 


Ctroar Of  CO'T'Put.f  Sctef'ce^ 

Carnegte-KteHori  Untversit/,  Pitisbufg*^,  Pa.  15213 


abstract 

A ae^cr  piion  of  *976.  version  of  Ibe 

Hcar5dy-ll  system  is  given  a!  the  Knowiedge>sourca  level, 
indicating  the  act.ons  Of  eacn  Knowieogc-source  and  lhe*r 

interactions 


!N’TftC2wC":CM 

Tne  ►^ariay-Ji  system  nat  oeen  aescr.bec  elsewhere  m 
terms  ol  its  systeih  Or garujaT.Qi,  ;nf.uO-ng  the  rrocJe*  wn»ch  has 
driven  that  design  [LeKs75,  ErMu75,  FePa76J.  Also,  the 
■ndividoa'  Knowledge  sogrees  ^«ve  been  separately 

reported  on  ,n  oetaii.  In  this  paper,  a oeser-pi-on  of  the 
Scpien'rx'f,  ;0/6.  version  of  tne  SfStem  «s  jiven  in  tern'S  Of 
the  functions  ana  interactions  Of  the  K$  ^ Th*s  does  not 

• nciude  a descr  u^ion  of  how  this  configura*;on  ■$  realized 
within  the  general  Hearsay  mode>  and  Hearsay''!!  system,  nor 
does  it  include  many  details  of  the  inner  wrsrkmgs  ot  the  KSs. 
Or  comparisons  of  Hcarsay-II  with  any  ther  systems 

The  tat'n  for  the  system  is  to  answer  questions  about 
and  rctr:e\e  documents  from  a coicction  of  computer  science 
aosiracts  (m  the  area  of  artifecia'-  ‘nteihgence).  Eirample 
sentences  ate. 

""xVhicn  absiracts  refer  to  theory  of  conputalfon^’ 

"List  those  a'ticles  " 

'*Whal  hac  M nsKy  'written  s.nce  nineteen  seventy-tour’” 

The  vocabulary  contains  iOi.  woros  lin  whrch  eacn  extended 
fcrni  Of  a root,  e g , the  plurai  ot  a no^n,  15  counted  separately, 
•{  if  aopearj).  ’’■'.e  grammar  whicn  octinjs  the  Icga'  sentences 

• > coniext  tree  and  mciudes  recurs*Of>.  The  style  o*  the 
gra.T/r.’.ar  1$  5wCh  that  there  a'o  many  more  non-terminats  than 
m conventional  syntactic  gf an^mars,  toe  information  cOPlaineO 
■ n the  greater  number  ot  noaes  provides  semantic  and 
pragmatic  constraint  within  the  grammatical  structure  For 
e»amplc,  m puce  of  ‘Noufi’  'n  a "Onvenlional  grammar,  this 
grammar  mcluoes  SvCh  non-lermioa  5 as  ‘Topic’,  ’Author’,  ’Year , 
'Pubiisiicr’,  etc 

Tile  g.'a'Timar  ai-0*s  c^cn  -ACrd.  on  the  average,  \0  be 

• Ci'o  *-co  by  . everteen  olr.e'  wo.'cs  of  Jne  vocabu‘ar’y  »he 


j Ti^r.  wc'k  WAS  sopporiec  0\.  the  Defense  Advanced 
^rc.ccts  “p.encv  iF«3462C-^3-C  *00  7«3)  ar^d  is 
“^on  loreo  ly  •''e  Air  Force  Ctt'ce  of  Sc  ent>lic  Research. 

2 Henceforth.,  a’  ocsc' ot-ons  are  vneerstood  to  aoc»y  to  the 
Sepie'^ncr,  1976.  s>s'em 


stanefaro  deviation  of  tnis  measure  is  very  high  (about  51). 
Since  some  weds  can  oe  toiiowed  by  many  others  (up  to  300 
in  seve.'ai  cases)  For  ’he  sentences  used  for  pertormar'ce 
testing,  the  average  iengtn  is  seven  words  and  the  average 
nunibnr  ot  woros  that  can  follow  any  initial  portion  of  the 
sentence  -S  thirty -<our 

T.he  Sepienibcr,  •9'^6,  configuration  of  the  system 
recognizes  about  ot  »ts  test  utterances  (ryn  biiryj;  word- 
for-wc'd  correctly,  with  auout  902  Of  tne  utterances  dcng 
interpfciec  semant.cany  correct. 

SiG'NiVi.  ^■COU:f:’!ON.  PARAIoETCR  ExIPACTiO-W 
AT iCN,  anci  I ABFlLiN'u 

An  mpwt  utlerance  is  spexen  mto  a med'um'3ud-ty 
EJectro-Voicc  R£-5i  ciose-speaMng  headset  microphone  m a 
faifiy  noicy  environment  {>65  db).  Tne  audio  signal  is  iow- 
passed  fi.tered  and  9-bii  sampico  at  1C  KHz.  A'l  subsequent 
process'n;^.  as  we.i  as  contreii  ng  the  A/0  converter,  '$  digital 
and  •$  done  on  a t;me-sharod  FOR-IO  computer  Four 
paramelcis  (ca'Ied  “ZARCASh*)  are  derived  by  simple? 
agorithns  operating  directly  on  the  sampled  signal  (Go2a7T] 
These  pararueters  are  extracted  in  real-time  and  are  initially 
used  to  deteci  the  beg-r.mng  and  end  of  the  utterance 

T.-'e  ZAPCASH  parameters  ai e next  used  by  Ins  5rC 
Hnowieoge -source  as  tre  oasis  tor  an  acoustic  segmentation 
and  Classification  o*  the  utterance  This  segmentation  is 
accomphshoo  by  an  iterative  refinement  technique;  First, 
5 icnce  s separated  from  non-silence;  then,  the  non*silcnce  is 
broKen  down  mto  the  sOnOrar!  and  non-sOnOranl  'BgtOns.  elC. 
Eventually,  f./c  ciasscs  ot  segmants  a-e  produced:  snencs. 
senorant  peaK,  sonorani  non-peak,  fricative,  and  flap 
Associated  with  eacn  ciassifed  segment  is  its  duration, 
aotoiule  arpi-iuGo,  ano  amplitude  relative  >0  its  neighboring 
segments  ».  e , ocai  peak,  locai  vaiue.  or  plateau)  The 
segme'-ts  are  contiguous  J!nd  rTqn-Overtjpping,  with  one  class 
oevisnation  for  each. 

F-naU;.  the  StD  K5  does  a finer  labe'iirig  of  each 
segment  The  laoeis  a^e  diiopho'‘ic-!ihe,  jnerp  are  cu'^cnii.  98 
o;  them  Each  of  the  98  *aoeiS  -s  def'neo  cy  a '-ector  of  auto- 
corrotaticn  coetfic  en;s  [i‘M-7E).  These  templates  ^*'’0 
generated  iron-  soeaHer-oependent  tra-rmg  data  that  have 
been  nana- abeiieo  The  result  of  the  labeiimB  process,  wnich 
-motches  the  cerlrai  portion  of  eacn  segment  aganst  each  o* 
ihe  Ter.piDies  us  ng  me  itanura  mcir  c,  is  a vector  ul  9S 
n^moers,  the  ■ t^*  ngn’OCr  -S  ar*.  est-maf«  Of  the  {.negalivC  ‘Ogl 
p-obaol'ty  that  the  segment  'corpsenis  ar^  occurrence  of  tre 
I'th  a'.'onhone  n tne  .ace.  set. 


, 799-802. 
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rtuCi®‘  and  !be''  Out-^ard  tfac*^  nuc  ^os  Th* 

syUabJe-cass  .$  ariven  pv  a orObaO'i'St-c  0< 

”s y iiaDic* -( ■a<i  •*  scf;r*ent'’  pf Cdocr-O»'s,  rulas  a^-cl 

o/'Obao'iil  a/'?  'ea'^ed  by  ao  o»‘-'r.e  pfo^^am  »fcn»ch  s 

:fdir»«io  or  -'.ar.o-.acp*  ed  (Tr^  c jf''ur»{  ?fa-r»'^.g. 

xn  ^n  .r.  ssca-^c- 'Ccpe^oen!.  -s  Q'.c^  6D  TOntaninj 

aoout  360  wQj’C  JCKeni.)  fo'  eacf^  n^cie’-^s  poiiJiO-r,  savera; 
competing  c .•i*aoie-c!ass  hypol^eles  a^'e  generated  ■••  typicai»y 
three  10  e'ghi 

The  syi  abio  c ass-es  are  used  to  h,oolhef'?e  words 
Eacn  ot  tre  !0*l  wO'ds  .r  !he  vocab.Md'y  s spccihed  by  a 
pron-jnciatiOh  oescr:p’*ar..  Fc'  *Ofd  nypothcst^aiiOh  purposes, 
an  nvorted  for'»  Of  the  dicliOi^ary  'S  ^ept.  »n  <Ahi:fi  fne'e  -s 
assOC'atea  wifh  eat:;  sy-aPie-cass  a*'  the  *oras  x-^-:n  ha.'e 
some  ororoAc-dt  Oa  cortaih  og  trat  sy  ao-c-c  ass  Ti-e  *-*0.*  KS 
'S^We'^Oj  ccK<  V.P  eacn  ryootnesTec  svfao-e  c *ss  and 
generates  wera  :ano*''v';ies  tro"'  amorg  tnosc  *orav  -.  onta  ning 
that  s ,Mabfc*class  For  each  wO'd  ‘ra;  is  vu.ji-s,- •<••-•.-:.  r o* 
•re  S . aO’CS  m on?  o!  the  pron-^eciai  Ors  must  mate  ■ aoo'-e  A 
threshold,  ^yp'cai  v.  oO  wO’Cv  0*  tee  ‘.Oii'Acrr.  .ocabu  a-'/ 
are  geneiateo  a;  c?;^'  sv''aoic  n .trei'.  rc;  tr'' 

''na*;y,  fro  ge'eratea  word  ca'".oidales  i'e  ra*cd  a**<j 
the  * begiA-  ano  eoa-iir'es  aa;c:siea  by  the  iy//--KC 
r-nowi-cigo  source  [)vWVVo77i  ro'  each  woro  n the 
'.'jcaOv.ia',.  WIZARD  has  a netwO'K  wpcs  aesc'-bes  l^e 
possible  pronuAciat'Ons  This  '3t'Ag  s ca-Cyiated  by  tir.dihg 
the  oath  Ih''0u6h  the  networi^  which  cest  matches  t^e  laoe  ed 
segments,  using  the  0 stances  assoc  ated  wth  eacn  abe-  ter 
?£<!■>  seL'^’Chl,  the  rating  .V  Irer.  z.-.cd  or.  the  d ‘'C'encr 

between  !h,s  best  path  and  fr-c  seg*’-''!  aces' 

Tne  result  ot  me  word  p'ocess  nj  so  tar  .s  r set  o* 

words  E;»ch  v-o'd  .nciudes  a beg  *•  ’ ^c.  an  eno-‘  ''n.  ar.n  a 

iOn^ocA'^e  ra!  n{j  A po 'C>  ^S.  ed  sORO'CT,  t .a'a 

conifc*’’,  celeds  a suS  -rt  ol  these  xO'os.  b'l  .'o  -’a  •h,',f  '-'-.•.cs 
and  'at.ngs,  to  oe  fNCCtrcs.^ec.  t .$  these  ceectevt  wo*-i 
hypoineses  tr^t  IC"-!  moc’;  0*  tre  base  tor  li’c  lop'e'*o 
process'n^  tr,»‘  rox  ceg '-S  - z fhese  sefectea 

'’  .pojnnse^  •'*c'  iPc  ai'Ocf  T®,  * c*  :•••.•  -krp'j^  ^'tua'-  *-pc*en 
■ o,  *cor'eci  xC'J  , pet' 'csfr "'j  xifn  eac-**  co*'rc:t 
hyoothes  S ha..''g  a wn-c'  t O'*  !nc  avn'Jge  J00i:t 

'nree.  as  C0'''narec  to  tne  '-ve  fO  Iwenty-f-ve  0“  SO 
r.ypotf'cses  wn  :orr::e;e  * tn  .*  e.  wren  s.gri-».:an|i  / 
overlap  t I me  't-e  rcr-seecec  *rros  a'e  reta  neo 
mlcrna  iy  by  VvC^C-CTL  fC'  possib  c ate'  n/po!hes  za*-on 


J n jfct.  a r-  •'  9'j‘e  ve'-  on  Ct  t»~e  “•‘‘•R’"'' 

sDce::.  rccor*.  t c''  SrS!e"»  e«-ec'  trat  ! ''at  0**® 

»-«r  ^ tnr  w^'C  'A'ne'  t'  3^  O'-B  retwbr^  wlr.  a • 

*orvv.  any  cc''fe''ccs 


',Vb r C__ *■  Gr  *■  - ^ n 

Tne  ^r.ox  eopc  source  [;.e5e77]  nas  the  jOp  ot 

ge.ncf  ati-g,  trem  fne  wr,ro  r-./poiheses  ge''e*'ateo  boltor'-up,  a 
sn’ali  set  fh-re  *o  >n)  ot  wcro  sequence  hypotheses 

Each  of  these  sequences,  c*’  tiLands.  can  be  used  as  the  bas's 

for  OiOant  or'  into  •arr/''  isiands,  hopetv-iy  Culminating  m an 
n^poihrs  c If  spa -.s  f t'  entire  uiterance  Moiti-word  lS^anci^ 
usea  rAmef  t.-.an  s r ..e-word  s.ands  because  of  the 
re.as  ue* . poor  -ei-ao  o'  rat  .ngs  cf  '...nigie  words  as  weii  as 

the  ii.'-^'ifej  syptpctic  cor',i'a.nt  supp.-cc  Dy  sing.e  words 

WCSEQ  uses  two  Kinds  o*  Knowieege  to  generate  nuUi- 
woro  isianJs 

A idDfe  der-.ed  irom  the  grammar  ir\dicates  tor 
ev'if>  Orce;  oo  pa  r of  wOrcs  m the  vocabulary  (iOl  i « 
iOlil  wheiner  that  pai.'  car  occu'  in  that  Order  m 
scr-e  serferce  ot  the  detmed  a.nguage  Tpijs  binary 
lai>«e  <>.h-:h  contai''?  about  177  thus  defines 

■'ianfua^e  aOtacency" 

AcOjst-c  pnonetic  Kncw.edge,  e.mooo  ed  m the 
f<S  IS  »pp  eu  to  pairs  cl  wore  Hypotheses  ana 
• s ‘.o  oec.fle  >>  that  pa  r might  ce  cons  cefed  to  be 
t.*  c-dd.jccn:  n tne  utterance  jwtjCT  jses  the 
n-rtonar,  at  ons  ana  e»a*i-ncs  ^"e  <e{^mcnro  at 

.uhCt  f(*  lAj"  0''Cr'ap)  m n ar  rg  its  'Jec-s-O*' 

wOuEC  ‘-^^es  ’.^c  higrest-ratcd  smgie  words  an<i 

generates  ~*u:!  -AOra  seo-entes  by  expanding  them  with  other 
A>po:rcsjed  woros  tnat  are  oolh  t.fne-  ano  language- 
ac,.>ce'--  Trs  e»&anso'-  .s  contronco  0/  heurist.cs  based  on 
the  Hi,'' ;:rf  ano  ra:.i./,s  of  competing  word  hypotheses  The 
Oh'^t  0-  these  A-pros  seouences  (which  occasionally  inc  udes 
single  xoros;  arc  nypolresizeo 

T'y  tcp-c-'-d  e'0ccs5''g  s starteo  by  tne  c'eaticn  of 
t'*esc  wO'C'seauence  -.aotneses.  SubscQuentiy,  WOSEC  ‘■ay 
ge-.e'i'te  accitiO-a'  n»ootheses  ft  the  recognition  process 
tve''’&  "oi  to  be  maK  .n?  progress  paced  on  Ihpse  a-^eaoy 
r^>  ;:otf  os-/cd  Tneso  ado-t  ona-  hypotheses  may  i''.cluae 
Shorte*.  cc-ccmpose.i  ver:-ons  of  some  of  tne  or.jmal  ones 


CC'  dwve  the  sv'\-*t*  C constraints  used  in  the  g'  ‘ on 
0'  fre  <*Crn  I'quencec  a.'e  on;-,  pair -wive,  a sequence  iO~ger 
fr^n  two  worcs  n^ay  -ol  DC  5'/ At ac t iciliy  acceptao'e  A 
sD'-DC'cn'  p!  tf  e 5-T“  |»*aSy'*T,  KaLn'"’]  Kr-owiedge  source 
T..'«r^  pa'’*:?  ? wr  -fl  seu-.e-nce  cf  a'P'tra',  eng:n,  using  the  ti.-;; 
cc.-slran:-  g.  by  :-p  language  Tr,5  pars-ng  does  not 
require-  ti^a!  1^0  word  sequence  fo’n-  a coiT^pcfe  non-termnal 
■n  the  ,'r>'"pia.'  r-or  tr  A'  in':  sequence  be  se'“tenct*  nit.ai  or 
• en»e''  c ■ na  . emy  t-,4'  t-'e  words  occur  cor.t  gucudy 
VC *11  'p  r.  spi-e  zer‘e~.ce  o*  the  la^.'wage  M a sequence 
.lutr'e*:  s r.per  net  pa'ce,  *-e  hypetnes.:  ■:  marked  as 
■ e:ec’er/  C';r?'wice.  a p'-rase  n . coines  s s c'eated 
^jSOc  a!’*:  w :h  're  pr'^^e  nycufhps.^  ^iOrq  sequence  of 

w-  rf.  : s cD-mpC-ec;  ^*s  xe>i  as  -.IprmaliOn  about  the  way  ter 
wa/S'  the  wqrjs  parseo 


iS 


Wo^d  Pfea-fi  or.s  ?fo*n  P^ra<«*s 

Anot.’C'^  CO- pO’^C'^.t  z*  ;r-e  S^SS  ^•'.ow  ecge  source  Ci»n. 
tor  ph>  i'.e  •'.ypcJr.esis.  ser^ersfe  predsct-orts  oi  an  words 

•*nich  cat!  r-.n-c-Oijtei*’  pre-'Cce  a'KJ  a.  whscn  car»  ;r*’'r'e2iate'> 
totiow  the  p-irase  -n  Trc  language  l"k  ciCirvg  tp'e  cor^PutatiOr'  ‘c 

generate  p'^ec'ct-'Or’s.  th.s  <S  uses  the  oarsin^ 

infor '•laliOh  ajt.!C:-«a  to  the  pnrase  hyoc!hes*s  oy  the  pariirtg 
component . 

Word  Ver.t-cat.c- 

An  atier  O'  .s  r^iace  tc  .e'  *y  "r.e  e*  ctenc?  z*  o*  reject 
each  S'jch  prcc'.ted  word,  in  the  content  ot  «ts  preflictmg 
pncasc  vec'heo,  a conf'Oer.ce  ratirg  tor  tr-e  ^worfl  most  also 
oe  generated.  Frst,  •{  wora  has  been  hypotnesi^ea 

prcvroiis’v  and  p.^cse*.  the  test  tor  t**‘-.e-aOiacency  (by  the 
xS).  it  :s  marKen  as  '-er.f.eo  anc  tne  word  nypotnes'S  -s 
associated  with  tne  prediction  (N-cte  t^at  a smg'e  word  may 
thus  d!'cc.''te  assocralej  w.!h  ♦.evc'al  c.f'ercn!  phrases) 
Scv'onj.  a '..earv;>  .s  n'.^ce  ot  !n;»  .r.te*’''a-  stc  e o»  AC-*^0-CTc  to 
‘ee  it  thp  cann.n.Hle  - an  be  '■•atchoo  o.  a p'.''^oos  -•  f,“''erjteo 
canrj.-jte  *n:cn  nao  '•01  ceen  n/pot'^esizec  A^am. 
rraKfis  a .cC^'T-ent  accut  I;’'»’e*d3irtcen' / ^ina>./.  WIZAkO 

COr'pa.'fs  its  wOrO-prOn.^ricalO''  ~et*'''OrR  to  the  ceg“'er'ts  n 
an  alte'-’p!  to  verity  tt'e  prcd'ct'Ori 

For  eac  ' Of  '-ere  niMe'cnt  ^ •-:s  ct  ver  f jre 

aDpro<  rna‘-  l>e^r.-,  (erc*-*.  ri  pi  *:••»  «n*o  oe*ns  r'cO'Cten 

to  tre  ^;ief:)  0‘  ’ho  pn^-ase  ts  t^^en  to  be  tne  end-t^me 
(Deg'n»t;me!  0'  Ire  ph'ase  ' eno-t  me  ^Oer^  n*Sime}  of  tre 
prcd-rted  wora  s rct  *inov.n  .'r'sa  -n  mcj.  or^e  '•eoj-rement  ot 
the  ver  ficalion  step  ■$  to  ger.era:e  '»n  appron-mate  end-l'mo 
(cegin-!imj)  tor  tne  verified  woro  Ir^  gene'ar.  several 
o.ftercni  ’'veriiors”  ct  the  were  may  be  generated  which 
d ?*cr  pr-mar-w  n ‘here  erd-t-rns,  s nee  no  content  tc  fh© 
r-nh!  deft)  o!  tre  preC'Cted  woro  s g'ven^  severa'  efferent 
estimates  cf  the  sno  •ocgir'.nng;  o'  t*'t>  woro  rr.av  oe  pjautiote 
oaseo  so'oiy  on  rne  segrrerta  cn 


jjwofantfC  t'.at  'ne  - r;;  s^ch  compicle  Spann. ng  hypoincsis 
w..:  .-.a.t  l/.e  .-..ghest  ralihg  0^  a‘f  pPSs  at©  spanning 
'''o-'c c n,  }>o;fcsfis  s-i!  PC  fCw^d  if  tne  search  were 

ai>Oj.cd  to  con*i''''..e,  so  ir-e  system  does  no*  jusi  stop  w.tn  the 
*'-t  Qr-o  r , tr  p t ic  :erist..' s oi  soc*i  an 

n>  .-i'C  . 'Pii  in  ;-r:,re  I'Om  furircr  COnsioe-' at  .on  Oti  or 

’lyootr  e'es  w.  ' \ oecaose  of  tre.r  iow  ratir^gs.  are 
jni.Keiy  lo  oe  pxtenuajc  into  spannng  hypotheses  wiln 
'al-r>;>s  Jiigner  trian  ir-.p  ;;}est  a^ready-OiSCOvercd  spar^n.r'.g 
sentence.  Th'S  •■'euristi:  oruning  procecure  is  based  on  the 
fOrf.'  of  trvs  rat  r.p$  li.nct.O"  hOw  tne  rating  0’  til©  pnrase 

•s  oer  a from  t'.  const. t^ent  words/-  ^ne  prunmg  proceowre 
con5«cers  each  pari.ai  pnrase  anc  uses  the  ratings  of  other 
a/C'3  n;r-0!'ieses  in  '.-c  t.-r-e  areas  not  covered  py  the  pnrase 
•0  aeterr-me  tne  phrase  m.;ght  oe  extendable  to  a p-irase 
rated  r.ghcr  than  the  soann.ng  hypothesis,  *f  not,  the  partial 
pnrase  s urureci  Tn:-;  p'ur-ng  process  arxl  tr©  rating  and 
ha-ting  pC'iCies  a-e  d:scus:ed  in  [KaPo??]. 

The  recojh.f-pr  pro-iessing  fmaily  raits  m one  of  two 
ways:  F,fs».  tr^erc  may  o©  no  mOre  psrt.al  hypotheses  left  to 
to--"--oc'’  for  p-'c’Cict--'?  and  extenC'n.g  Because  of  the 
C Or-rS'natcr  c y pf  ire  gram.^^ar  aid  tn©  i-Kelihcod  Of  finding 
'jj-'c  prcuict  O'*  ‘.nal  i',  rateo  a!  ica:i  above  th.e  ahsoiuie 
re.**  :'  O'-  Mires-  c.  j.  (r-:.  ‘cr-’  of  ferr-  raf-Cn  happens  -.r.en  the 
pr^rt.nr.  jroceJ-'C  hac  ;>ten  ef'ective  and  nas  eliminated  all 
compel  iory  Secpro.  ire  etpenO'tu.c  of  a predefined  arnount 
0*  ccmoujing  rr...Owrf?s  (tm©  Or  Space)  a. SO  halls  the 
r(*v  p/''’'’*  I'--  ^lit-.a*  t"f-^nr>lds  -.,'-'0  ?'e  sol 

accCfo.n^  -p  ?-••  w.-'-ihce  or  tre  sy‘:em  cn  s-iiijr 

‘.o.''tences  •■/».  0!  'he  g-’en  length  ar\d  over  the  same 
• Oc  auu'a'/  and  i ar  j 

Cnee  ir.t*  'oco'hiion  process  s ha, ted.  a selection  of 
on©  or  more  ph.-aje  hypotheses  is  mace  to  represent  the 
fcs'..  i K a;  -c.  :t  on©  spann-ng  sehtence  hypothesis  'was 
:re  h.“  r.c  j:  •' ateo  t».cii  rypotnes-s  'S  chosen^  otherwise, 
a sc  r*ct  or  of  oe.cra  ot  the  n.ghest -ratoo  of  the  p.:.'f!ai 
pr.f-jvj  r^optroses  s •■•.ice,  biis.hg  the  se;e:5iOn  to  the  Or.gest 
Z 'CS  .>  i'  . • .c;''h  *;/  o-  I*  .ip  »:n  t mc)  the 


'.Vord«^h* »se  Concatenaf-cr 

For  each  vcf.f  co  word  anc  -is  p'oct-c-  ng  pnrase.  a r.f** 
.sra  -onger  chrase  may  e r--eraieo  T*' s process, 
.11 -0'’'P'i  '.r‘ea  0/  ♦.  'CmponenT  pt  5Ai5  s — *ar  to  ‘fe  WO'O- 
Seq^ence  'ecOiin;!  on  coroc-e-l.  T'/ZI  c'  pars  the  -wC'CS 
3f  the  cr'3.:.a.  pnras-a  a-..,^~r'"c3  .v  ‘“e  e'  ’ o.i  wor-j 

’ne  c«tendcd  pnrase  s t"-en  nypo:-~e$.reo  ano  '•(•joeit-  a 
rating  ^^ascd  on  !rc  raJ  r.gs  0'  ’ne  wOrcs  that  comOOS*'  •*,. 

Comn  ele  Ser!ft"ces  '^.o  -.*  C-  ‘r  » 


T*'e  tcp",r*3  p'p:‘'.c:ng  opera;  o-s  mc’-ucc  (a)  word- 
.'.-a-'c  gene.'a*  o'*,  '.b*  *0'a  sequence  parsm^,,  (c)  word 
c;ct::t'On  ifo-"'-  p*'.»sef.  'z)  wpro  VC'.*. t ea:. Oh,  and  te»  *-0'C- 
pr.-..»,,*  c 3;',  I'en.i*  •I*'.  . ' •‘‘CSC.  (o'l,  ^‘ /.  aro  (d-  are  *.hc  "'Or'f 

• c3..t  ->i  • '.t"’  ) ■'  ' • '•  . the'C  .i.'e  a “'.•.■njc-  of 

.* . * to  ci‘  per'e-' •'•'jr.  Af  .a.-ous  p.aces  n ihe 

jji  iP' .'•"cc  The  sect'  /'  each  poir:  n (ng  .rore<.s  r*r%  hf 
*k.r.icii  O'  i'*cse  act  or  - ;p  oerfori':  » proufem  of  con'r>,naloriC 
co'*‘''0'.  s-'c-?  *!-..■»  erec  on  0'  earn  a::-on  will,  n genera', 
gc'erate  .■'■ere  ‘iuv*'  a;;-0'’s  to  oe  don© 


Two  u”  Owe  “word"  h/octheset  a-e  gene'd'eo  before 
fhe  ! 'zi  aicJ  a‘’er  fhc  :as!  segment  c’  tne  utterance  to  de^o!** 
oe»-,in  and  end  of  u"e'ance.  'cscect've  / '"'esc  sa'^'e  wc*as 

O'p  r^iujed  -r  ■:'.e  svrtacic  scec’f  ca'-.on  o*  the  .ar.g..a;^e  a-.o 

, spc,*.'  as  ;-‘e  ’-rsl  and  .as'  ’er-n-'a  s o?  cvc'y  comoiele 
sen*e''cn.  T’-us.  a'*./  ve-'.*  cc  o*trase  ’“at  -rcn^o*^  *nese  as  Is 
-•Kt'eme  co‘'st  i'jer's  s a cc'^oeie  'er*e'‘c©  a^o  'oa^s  tne 
oii^rpncc  S's/C  1 a ser’e“re  oecm-es  a car.-:  date  tor 
•elect. or.  as  *.“e  systcr'-  s recO;,'*-  o'*  resut 


st * atei  ‘re  oo  **0* 


to  rant:©  s prcpem,  tr©  ryjarsayil  system  has  a 
t at  st'C a-  , r;,:;?a  fc  cCv-er  [baFo/rj  woic'i  caiCuiates  a 
C'  Or.'v  C'  ear:-,  aci  on  a .p  so  e-.*s,  at  each  time,  tne  wa  i-rig 
aclor  v*'-  're  .nT,nns'  or  0'  ty.  Tr©  Dr:0'i*y  calC  J'atiOn 
a'te-ri’-.  \z  e-.i.mAie  tre  ..sefp''ess  of  're  a:t.on  -n  ^utf  inj; 
;rc  c-r*a'  s- tiic--'.  y.-oa  ot  roccgrij  ;hc  otlcrance  The 
c at  cr  s or.  - 'Or  n'.alion  spec  t eO  wnen  the  act  on  s 

■•'  •ii-'Cd  -C'  2»a“-.p  r.  tre  wO'd  verfer  is  triggerpo 

^rert-  V'  wC'jj  A'©  p-r.^ctea  'rdm  a ph- ase  hypothesis,  tre 
n’C'-ial  cr  pac.eo  ic  '~o  sc  'ccuor  -n  C'der  to  “eip  ca'C.'a'e 
t**c  pr.or;^-  p‘  • -.  2 t a**' ai.cr  o*  i.re  ©r  fier  .nciudes  suci 


i i J 


as  tiT-'c  ana  t^e  orea  ctmjj  orrase  and 

nun'ber  n?  words  predicted  >.  aoc'*.  on  to  tte  acfion-spec<t»c 
• nfcr r»a! lon,  I'^c  sfi'eduier  Keeps  trac*  c?  the  ovei«'>'  state  ot 
sy^-'em  n ?e  'ts  oi  Ihe  kioos  anc  Q«-a'.'ty  of  hypotheses  «h 

eacri  l-rne  a^ea 


i>^'^rRPft£TATiO:v  ,-"<1  wc5PCr-.-G£ 

'he  SE^-^ANT  Knowtenge-source  [HaDi77]  accepts 
word  seQuehcevs)  re^yit  ot  tK®  recognition  process  and 
jerverales  an  interpretation  m an  cnamO«QuOuS  format  tor 
• r,ier.)Ci:on  w.ir.  tne  oata  base  tnat  the  speaker  is  querying 
nor  e^ampin.  ;he  spOKcn  seh'ehce 

'*Wnat  has  IvtnsKy  sir'ce  i97'5’'* 

■ s represented  m this  format  as 

Type:  CREQ^CST 

Suhlypc:  SG'jCRT!AUTKCR‘DA'^£ 

'Gate.  ii97C,  Ajtnor;  ■■M1\SKY~] 

The  -nterDrelation  •«  ccnstrccten  uy  actions  associated 
•Aith  "temanticaiij  ••-ercst.r^g■■  f'0'‘-lerK\r*aj5  m the  parse 
t'eeis)  of  the  recognised  sequencets).  If  recoghit-on  resuits  m 
: or  more  partia:  sequences.  ScMAVT  constructs  a 

cons-iten!  interpretation  basea  on  ir.  o*  tn®  part>ai  sentences, 
taung  >n;o  account  tar  each  paf?*<fi  sentence  its  rating, 
temporal  posi.'On,  ana  cons-siency  (or  compet^l. ^eness)  as 
comp-ircct  to  tne  Other  partia-  sentences 

*ne  aSCO  Kr.pwte<i5?-S0vr:e  ^naD7rj  accepts  the 
tcrmailea  mte'prctation  ot  SEWANT  anc  proouces  a response 
to  the  soea*’.®'’.  “’’his  response  s often  the  dispiay  ot  a 
se:ected  poriion  of  the  queried  cata  oasc  in  craer  to  retam  a 
cpnercht  :n*erprcfaliOn  across  sentences,  CISCO  has  a finite- 
state  Tiodei  of  tie  discourse  wr.ich  -s  updated  w-.ih  each 
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ABSTRACT 

In  Heafsay-H,  a word  recogn.jar  hypotnesizet  words 
boiforri'up  /rom  acousftc  data  Usuany  many  corr.oetmg 
words  are  hypotKes-zea  tor  each  f.me  interval  c*  speeco. 
with  Ihe  correct  word  rareiy  loo-raoReo.  Due  to  the 
unreliable  ratings  of  word^  ana  tne  '-m-ted  syntact'C 
con5.lramt  suppled  Oy  single  woroi.  tr.e  use  of  s:r^g•:e*wCfd 
islands  would  cause  the  reccgnition  syste*n  to  e*P'Ore  many 
blind  alleys  before  abaivdonmg  an  incorrect  rsiand.  In 
addition,  the  njit'plic-ty  of  words  rr;a>ves  the  parsing  of  ail 
posstbio  word  sequences  eKlreme^y  timc*consum;ng  The 
Searsayll  islarsd  selection  strategir  uses  (D  of 

what  word  adjacencies  are  allowed  by  the  grammar,  (2) 
analysis  of  acoustic  data  at  tr.e  ,uncture>  between  word 
hypotheses,  and  (3)  heuristics  based  on  the  nwmoer  of 
competing  wOrO  hypotheses,  tO  tOr«n  muiti-woro  iS'andS 
which  the  syntaK-lcvei  Kncwieoge  source  f'rst  checks  for 
grarnmaticaliy  and  then  attempts  to  evtena  to  form  • 
complete  recognition. 

IMTRQCUCTICN 

Conventional  s'rafe^  es  for  controlling  the  search  m a 
continuous  speech  understanding  system  fan  into  two  maior 
calegor'cs;  iefl-to*'  ^nt  ^M^spy  (iOwcr.'c.  19r6),  r^arsay-I 
[Peddy,  1973]'  ana  is:ar.a‘dr*wfc*n  (S^I  [Patton,  1975]., 
SPClllS  [Woods.  19751,  f-ea.-sayu  [lesser,  1975]) 
strategies.  In  (he  left-to-rs^ht  strategy,  as  the  narre 
imples,  the  search  a'ways  oeg;ns  at  the  start  c»  the 
utterance  and  cont.nucs  to  extend  -n  a 'eft-io-right  manner 
each  partially  hypcthes  zeo  pnrase  that  appears  p-aus-o'e. 
In  contrast,  the  »siand-crrv en  strare^y.  bciO'C  oeg-nr  r.g  me 
process  of  phrase  nypoihes  zaf  on  and  extens-cn.  'i-'st 
performs  a scan  of  (he  entire  uttera^'ce  ‘n  an  aitempt  to 
spot  likely  words  [Smith  1975,  *^ovs*ad  1976}.  ’ne  cest 
words  found  in  tr»5  phase  a*”*  closer  as  t^e  iri;*a.  p-rasai 
hypotheses  *0'’  fhe  second  pha'.e  of  the  searen.  ;n  this 
second  phase,  a partial  ohrase  cncsen  for  furtip'’  extensions 
can  be  extended  by  pi-edict  on  of  gra-^^at-ca-iy  'egai  word 
extensions  on  cfhcf  the  or  ' ^•'t  or  .n  both  o-ections. 
depc  ding,  for  instance,  on  (he  consirjnls  g.ven  by  fhe 


This  vork  was  suoported  in  ^art  bv  :he 
Defense  Advanced  Research  ?r-;erc$  .^genev 
( t^620- 7 )-C*CC74  » ar.d  is  5o;;;:orc<i  bv  tr.e 
Air  Force  Office  of  Scientific  Researcr. 


grammar  about  wh»ch  dmecfion  has  fewer  extensions  (see 
hfayes-Roth  and  Lests'  1376.  Paxton  and  Roomson  1975, 
and  ‘Woods  19‘^6  for  a discussion  of  techn.ques  for  choos'^'g 
the  next  hypothesis  to  extendy;  this  strategy  allows  !ne 
phrasal  hypothesis  to  be  concatenated  with  existing  partial 
phrases  to  construct  r\ew,  enlarged  nypotheses. 

Tne  advantages  of  the  lefl-to-rignt  strategy  over  the 
more  sDomstiCateo  is:ano-driven  strategy  are  mainly  m the 
area  of  etf-cicney  (1)  the  computationally  expensive  word- 
spotting  pnace  is  Bypassed  and  (2)  the  application  of 
grammatical  Knowtec^e  and  iho  Overhead  for  controJ-mg  the 
searc'i  •$  much  less  expensive.  The  major  disadvantage  of 
the  ieri-to-r-ght  scheme  'S  that  the  begmivrig  of  the 
wtterarce  may  not  contain  very  good  acoustic  data  ana  thus 

• oao  to  initial  word  prcQict  ons  (hat  are  very  poor;  m this 
case,  it  may  be  very  oifficuit  Or  impossible  (if  the  correct 
word  was  not  hypothesized)  to  recognize  the  utterance. 
Tn®  major  advantage  of  ifland-Ortven  strategy  *5  its 
robustness,  there  may  oe  hypothesized  more  than  one 
correct  initial  is'and.  and  thus  there  exists  more  than  ore 
seouence  of  steps  to  scn:ove  the  correct  reccgn.tion,  in 
aodition.  the  isiand’d'’iven  strategy  wouid  seem  to  have  a 
higher  proDapi'ity  of  starting  the  search  with  an  imtiji  itiano 
trat  ic  va'i.o  because  of  its  word>spotling  pnast  However, 
th.s  word-^poti-rg  search  may  not  ift  practice  produce 
results  as  vaiio  as  would  be  expected  because  words  are 
predicted  based  only  on  acoustic  constraints,  n«itne.' 
grammat.c  npr  co-a'tic jiation  conslramti  are  used  except  at 
the  beginning  anp  end  of  the  utterance.  Another  aovantage 
of  the  isi j"d-d'iven  strategy  is  that  ii  can  use  vafat  ons  m 
the  branching  factor  of  the  grammar  at  diflerent  po-nfs  in 
the  utterance  to  reduce  the  space  neeaed  to  be  searched 

The  major  disadvantage  of  both  of  these  search 
strategies  is  that  they  are  particula'-'y  sensitive  tc  major 
rating  'er-ors”  on  smg  e words--cases  where  a va-d  word 

• s rated  ipwcr  than  an  invalid  word  m the  same  time  area.  If 
the  correct  word  m the  starting  area  -s  very  poorly  rated,  a 
best-first  seare.h  from  the  higher*ra:^d  alternatives  wni 
ceveipp  a very  large  search  space,  ano  tacxtracking  aH  the 
way  to  the  initial  ncor'pct  decision  win  be  very  expensive 
and  unlikely 

Two  means  of  overcoming  this  shortcoming  e>!St. 
Fi.'st,  m the  iimited-oroaoth-firot  searen,  tne  iN  fcp  rated 
words  in  an  area  are  used  to  oegm  searches,  and  as  'z^i  as 
one  of  thjie  S correct,  recognition  ■%  rot  prcciudeo  Tn# 
secono  a ternjtive  'S  to  dentify  muii--word  sequences  Of 
word  hypotheses  that  are  most  p'ouaoiy  correc*  as  (he 
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♦ ♦arting  'Sltnd)  m ««  S’^atery.  In  CO*r*5*rison 

with  Single'wOfO  liijriOi  O^  *e»  ^ *JO-» '6^*1  S-n^ie-wo^d  starting 
hjKpOthesas,  muiti'wdrd  $tfcjuC'*-Ct9  a't  more  rc»AD>^  tor  two 
reasons:  under  certa  n gcnt.'ra;:y  ao(‘iicao>c  coftOit;ons.  tha 
credibility  of  a seouance  ^^yoo'in-es  s erceeos  t^at  of  a singie 
word  bypofheSiS  and.  seccno.y,  the  renaoiiity  c!  a vatiQitv 
rating  for  a sequence  •%  grea’cr  than  fnat  of  a single  word 
hypothesis. 

To  substantiate  this  conjecture,  consider  the  following 
autfrage  rank  order  statistics  for  initial  islands  based  on  the 
three  different  approaches,  "hese  data  were  collected  ower 
34  training  utterances,  with  each  island  generation  strategy 
applied  to  all  cfterar\ces  The  average  sentence  tength  was 

5.5  words  Tr.c  iett-tO-rignt  and  the  S'ng  e-wpro  iS’and- 
driven  strategies  have  the  san>c  raru  order  statistic  which  is 

2.6  (i.e.,  there  are  on  the  average  i.6  is  arxes  wilh  ratings 
better  than  the  correct  or'e)  Jt 's  interesting  to  r.ote  that  m 
non#  of  the  34  utterances  d>a  the  ic^-to-ngnt  st.-ategy  f^ot 
hypothesize  the  correct  -word  m the  mitiai  utterarsce 
position;  ^Se  average  nur*'oer  of  words  .hypolhes:zea  for  the 
initial  position  was  eleven  words.  The  overage  rank  c'^der 
statistic  for  the  rr.jiti'  .ora  15  anc  strategy,  if  cut  uUeraixe 
IS  etirnmated  m which  the  rank  orcer  is  3C.  is  2.Cj  the 
average  length  of  the  best  correct  muiti-word  isUna  is  2.3 
words,  where  the  average  numoer  correct  words 
hypothesized  bdttom^up  is  3 0; 

A fAULTf-LCVEL  ^^PATEGY 

The  strategy  found  to  be  most  effective  ir.  the 
Mearsayll  system  las  applied  to  a iOOO-woro  vocao-iiary 
with  an  average  word  fanoul  of  33f  is  to  seiect  mu‘ti*word 
sequences  of  word  hypotheses  as  starting  islands  for 
synta»-levei  processing.  This  strategy  ‘ntrocuces  a new 
level  of  hypothesis,  the  word  sequence,  between  the 
conventional  lexical  and  pnrase  levels.  A word*seque«ce 
hypothesis  ts  a COrvcatenaC'On  or  One  or  more  wC*d 
hypotheses.  In  contrast  with  a phrasal  hypntnesis.  a 
sequence  hypothesis  >$  created  oerore  the  svotaa*ievet 
knowlecfge  source  begins  its  work,  and  may  not  be 
gram.matical  (re.  »t  may  represent  a sequence  of  words 
which  does  not  appear  m any  sentence  <n  the  language 
defined  by  the  grammar). 

The  decision  to  create  word«sequence  nypo»heses 
arose  from  the  realization  th^i  the  como.r.aicna:  space  of  aii 
possiOic  sequences  of  word  hycotheses.  generated  as  a 
result  of  the  word-soottmg  scan,  c*n  oe  reduced  sni-ply  by 
applying  a computationally  snetp-rnsive  f- ter  to  tre  data. 
Th»s  filter  IS  based  on  simp'-  hihos  of  gram-jiat-cai  anq  co- 
articulation  knowledge  about  wnich  word  pairs  a'C  pOss>b>e. 
The  grammatical  constraints  are  specif-ed  hrougn  a square 
bit  matrix,  whose  order  is  the  stze  of  the  vccaou<ar/;  each 
•ntry  <»,;)  in  the  matri*  indicates  wneihe.'  > o.'d  j can  follow 
-word  I in  the  gramrnar  !f  two  words  can  foiiOw  ea<n  otfier. 
they  are  called  "language-adiacent*  The  co-arricu»ation 
constraints  are  specified  Inroc^h  another  sqoa'e  matri*. 
whose  order  iS  the  S'Ze  of  the  number  o'  phonemes.  Each 
entry  <i,j)  in  the  matrix  mqicates  what  tyCC  Of  aebustic 
segments  are  a'loweo  n the  juncture  bei-^een  l-wo  words, 
the  first  word  eno.ng  w»lh  the  phoneme  i and  the  second 
word  beginning  with  phoneme  j.  The  append'*  contains  a 
more  detailed  ocscr-ptiori  of  how  the  co-a'tiCu'ation 
constraints  are  implemented.  H two  words  pass  these  co*- 


articulation  conslri  nls,  they  arc  caid  to  be  “time-auiacent" 

A wOrd-SequOAce  hypOti-CS-S  iiwflyS  COr.s-SiS  0?  wprd 
hy  po'hesrs  »*  hich  are  pair-wse  language-adjacent  ap»d 
tiime  -adjaceni. 

Consider  a pair  of  word  hypotheses  that  are 
language-  and  lime-adjacent  If  there  is  a third  hypothesis 
that  iS  iingua^e-  and  lim#-aojacer^t,  either  to  the  'cff  Qf  the 
first  word  Of  the  pa*r  or  to  the  ri^nt  of  the  second,  it  can  be 
concatenated  or^to  the  p»«r  to  form  3 three  word  hypothecs 
This  action  of  eviendng  couid  be  repeated  {leftward  and 
rightward)  until  'here  were  no  more  DOSfibte  estens  ons  [f 
there  were  many  j'ternalive  extensioni  at  each  comt,  this 
process  wOiiid  result  .n  th*  creation  of  a larger  numocr  Of 
partieiJy  similar  word  sequences  '‘-ewe-^er.  it  is  clear  that  a 
sequence  ot  more  than  two  weras  may  not  oe  gran.maticai, 
since  language-adjacency  is  oefmed  onty  between 
successive  two  word  pairs  The  Cetermmation  of  the 
grammaticaiity  Of  a sequence  by  the  syntak-^ve^  Knowleoge 
source  >s  a relatively  expensive  operation  (between  .1  and  1 
seconds  on  a PDP-IO  ><A10y,  thus,  there  li  a bias  aga  nsl 
creating  word  sequences  wtxch  have  a high  prpbabil.ty  of 
bc.ng  M,cC»rrcct, 

The  factors  which  are  of  ’ntrresf  m decidng  wnether 
a word  seQ-jencf*  is  good  are  the  ‘f'z*  ■ -■*'  'he  sequent?,  fr.e 
ratings  ot  its  irraivid’jai  word  hvpo;hcse5.  and  the  hum.;ef  of 
Other  word  hypotheses  competing  (o'/e.-'laopirtg  m fim,?)  -with 
each  of  them.  The  best  starting  island  is  (he  'ongest  one 
which  has  a very  high  probability  of  being  correct,  with 
correctness  taMrg  precedence  over  length;  correctness  *5  a 
function  of  both  incfvrouaf  word  vaiidity  rating  and  the  lack 
of  Similar  alternative  sequer^ts.  These  considerations  led 
to  the  fotlowtng  algontnm  for  sequence  creation: 

(1)  The  30  h ghest-^ated  word  hypothesei  anywhere 
in  the  utterance  are  chosen  as  initiai  one  word  sequences. 
Thise  with  ratings  -css  than  seme  cutoff  are  oisraroed 
unless  Q'Ag  so  wcuid  leave  less  than  live,  m wntch  case  (he 
five  tOu  v jrrs  a^e  Kept 

(2)  Each  initial  sequence  « assigned  a competing 
sequence  count  (CSC)  of  1. 

(3)  Eor  e>ch  current  sequence,  the  sets  of  ait  word 
hypotheses  lelt-<ngnt)  lancuaoe-  and  time-id(acent  to  the 
beginning  (enoing)  woros  Of  ihc  sequence  are  founo  !f  the 
current  sequence  has  C5C*.‘J.  and  ff  r-ght-adiacent  words 
are  found,  then  a ngnt  extension  wOuid  have  CSC"i>**a. 

(4)  Only  those  extens-ons  whose  average  word  rarmgs 
exceed  s cutoff  p-'oco-tionel  to  the  seuare  root  of  .'JiR  arc 
formed.  The  direction  chosen  fOr  extension  is  a function  of 
CSC  count  for  the  direction  and  the  validity  of  th^  h.^hest 
word  that  remairis  to  be  extended  m the  ipecif  c di.  ectior. 

(5)  Steps  3 and  4 are  rtpeared  m a recursive  manner 
until  no  mc'e  evtenstons  can  be  formed. 

Au  sequefKes  that  are  generated  as  a result  of  th-s 
process  which  arc  subsequences  of  another  seqi:eAce  are 
discardad 

• hi$  algorithm  produces  a large  nymper  of  potential 
word  seouences.  usuany  between  IC  and  iOC.  The  cost  of 
validating  them  all  for  grammaticaiity  's  expensive. 


gnot^'ef^  level  filtering  's  Dcr?c?^»ttcc,  ta'ea  on  ^ ra(«ng 
sttacKed  to  exet'  wo^d  ieoyence.  The  of  a seoueriee 

is  »r»  incr^'i'-ng  funft'Ort  Ol  these  CoarM.eS:  (1)  the 
dur«t.Or-wc  5>ted  awerace  word  AVG^ATE,  wOmO'jted 

by  suiri'1  ig  tne  proouct  (vyoro's  r d:ir^)»>.u'~bcf  of  syslab'ci 
if  cOr.f  ^-ns)  Over  aM  wo'Ss  ih  tne  seQuer*fe  af\<j  then  o vid  ria 
by  tnu*  njmder  of  Syllabici  -n  the  se^oervee;  i2/  fre  dyation. 
OUR.  cornpuled  as  the  perceni  of  the  oticrave's  syiiao^es 
cor.  airted  'n  Ih®  sequence;  (3)  the  nutiuer  of  words  m the 
sequence.  NWCROS  The  rating  fyr.ct:On  is 

rate  . AVGRATE  ♦ 0.1  * rM'CRDS  • AVGRATE  * O.S  • OuR 

The  highest  rated  word  seouence  pius  word 
sequences  whose  rating  is  some  epsiOr.  away  trom  the 
highest  are  chosen  as  candoates  for  further  evaluation.  In 
addition,  another  criteria  is  ervoiOyed  to  choose  sequences 
for  further  evaluation:  if  at  ail  potsipic.  there  snou^d  be  at 
least  or>e  word  sequerKe  for  each  area  of  utterance;  the 
time  areas  not  cohered  ty  tre  highest  rated  word- 
sequences  are  the  area:  that  arc  atterreted  to  oc  covered 
by  lesser  rated  word-seauences  Word  sequences  not 
chosen  by  this  filtering  are  rot  C'scardcd  but  rather  are 
held  in  abeyance  until  e<ther  processing  later  on  stagnates, 
or  an  eKistirig  word  sequence  caroidato  nas  oecn  found  to 
be  ungrawmalical  or  cannot  be  successfey  e»tencea;  »n 
these  cases,  these  pcorcr-rated  seauences  are 
hypothesized  for  consideration  oy  tne  rest  o^  the  system. 
Thii  process  of  word  sequence  generation  foi  th«  30 
utteraryces  results  m an  average  or  31  .ni»ia»  candidates. 
With  an  average  of  6.6  more  candidates  oemg  generateo 
during  the  run. 

The  basic  result  of  fh;t  a'gofithrr  is  tne  identification 
ot  sequences  of  t^r*ie •adjacent  and  'anguare-ac,acen;  words 
whose  crediDi-ly  is  h-gn.  Aithougn  a arge  proportion  of 
these  sequerices  may  not  be  grammatical,  very  few  h.ghesl* 
rated  sequences  arc  ever  inccrffct  fun-ess  no  success’ve 
correct  wore  pcrs  are  hvcotnes.jear.  r ..'tnermpre.  the 
computation  of  CSC  b>ase$  a^ai-nst  formng  p-g  sequences 
oecept  wnere  tre  chance  occurrenfe  of  a ‘ing^a'e-adiacent 
pair  IS  small;  thus,  in  only  ten  cercent  of  utterances  does  a 
highlyraled  'hcorred  cor.:ain  a correct 

subsequence  of  le-gth  greater  fnan  o-'c  whiC“  cocs  not 
occur  in  a longer  correct  sequer>ce  In  such  a case.  «f  the 
gramniaficalily  of  ihc  inco'rccl  o-g  scawC“*c?  5 re  ■?cl€d  oy 
the  syntax  Knowledge  source,  a oeco’^ros  t.on  of  tne 
sequence  into  two  maj.-ia.  s .b'CQjcrces  ccr.'s.  t.nese 
decomposit'or's  will  be  ‘lyi^Atnes  red  sw:;c~  / » .'ated 
sufficiently  n.gh  Ths  s a fC'ri  of  bacit'ac-iing  arsd, 
therefore,  'S  subject  to  me  san-e  weak.nesses  as  oiner 

bacKt''ack  ng  search  aigO'-thms  i-i  thi^  case,  however,  the 
probabii'Cy  of  a fa':e  iT'ia'  ‘s  ano  been  ^-eat  ^ '■eswced 
As  a result,  the  chance  of  a tofany  fruitless  search  is 

correspondingly  reouced 

"^he  effecti veness.  n terms  of  both  tefa’  system  error 
rate  and  amount  of  search  oerrc'm.ec.  of  ms  myri-word 
island  approach  Over  beth  the  -e’l'to-rigr.f  a“a  5 rr  e-worp 
island -driven  strateg  es  > ■''c  caieo  b,  fc-ow.r^  -.catistics; 

the  Overall  erre  r^te  fo*  t-e  tnree  stra'cg  cs  -s  67^  ATa 

and  5AT,  respectively,  in  f'e  'en  5«“'ehces  that  we'e 
recognireo  cor-ec!iy  by  a ‘ fnrge  slrale?  es.  *he  average 
number  of  prirases  hypotres-zca  ar»  A7,  68  and  68, 
respect.vefy 


CG:-;CLoSI^Iv 

TKe  mulli-woro  seqi,fr>c,  Eener.tion  onxtdijre  is  i 
ve/  rncwlpcfne  tOijrcei  .n  By  e.p^3:t,r,i  the 

■'-c'unaenty  o'  the  ianseje  to  .oent,(y  oiauvo:;  »ora 
seQue-.icc5  ana,  inf.aenia  ly  -crca-.ihs  the  B'0t>,bii-ty  lh,i  , 
>.31:0  slartihj  Iiiand  hypothesis  will  o«  more  hiPh.y  rated 
•han  an  insorrect  one.  this  source  of  anowledje  proaKles 
ycry  reliable  and  usatui  Krawietije  to  direct  the  overall 
searclv  In  our  oomion,  this  Knowtedge  source  ,s  a 
paradigrratie  eaampio  of  the  elfeclive  use  of  reound.ncy 
and  slat'f  ticaJ  sampling  to  achieve  a reduction  of  uncerta.nty 
■ n problems  characterized  Oy  fuzzy  arid  partial  miormation. 
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APPtMOlX 

Th4s  dppcricix  d«scr  £>et  th«  iwcro  pa  r adjacency  accep'ance 
procedure  (^’■jCT)  deveiooeo  ‘vafiiylU 
it  us«&,  an<l  th«  current  resut^  Such  a prOCtCh^re  ^nuSt  te 
COmDutattonal'y  irve«pen&ive,  ma^^irg  decstons  on  hunoreos 
of  pair)  of  hypothesised  woros  I!  «nutl  rcty  uoort 
k.r%Ow{ed$e  Of  word  junct'^es  anc  upon  the  intomalion 
C0ntair\ed  m the  se^rnental  transc'ipiton  of  the  spOKon 
utterifKe-  And  't  must  reject  a»  mar»y  ncocrect  patrt  (word 
peiri  not  actually  spoken)  as  posS'Dle.  wt^hopl  re;e<tir\2  any 
Of  tK«  correct  pairs 

As  inputt  receives  a pair  ot  word  hypotheses.  If  it 

determines,  bsoed  upon  times  associated  with  the 

hypo*^eser,  !‘'c  iu'':turs  rj’es  conta-rsed  the  provecure, 
end  the  segmental  oescriDtion  of  the  spoken  utterance*  that 
the  words  are  adjacent,  the  pa:'  is  accepted  as  a valid 
sequence;  otherwise  it  is  rejected 

The  word  junctures  upon  which  vUfjCT  must  maK.e  «ls 
decisions  fall  wifh«n  three  distmct  caces;  (1)  Tiine-cont*guOus 
hypotheses;  Words  which  are  hme  contguous  are 
immediately  accepteo  &/  uUfiCT  as  a possible  sequence;  no 
further  tests  for  aoiacency  are  performed  (T,  O-e* ;c.op.ng 
hypotheses:  W’hen  two  -oms  ove*5ao  in  t;me,  puncture  rules 
are  applied  m the  contcit  of  'he  segmemai  irw-rpreiat'Oc  of 
the  utterarsce  to  ceterm-ne  * sec''  a juncio'e  rs  ai'owaoie 
for  the  word  pa'r  (3)  Separated  ■'/potheses:  Wnen  the 
words  are  separated  by  some  in>‘-va;  r*  time,  ryifs  a>  e 
applied,  as  m the  overlap  case,  w determine  wrethcr  the 
pair  can  be  accepted  as  a vai>d  sequence  m the  utterance 

The  juncture  ru'es  used  by  JUfjCT  are  of  two  types:  (1) 
allowable  Overlaps  of  word  end-  and  beg-n-pKof^mes.  and 
(2)  tests  for  a sallowed  segments  w'th>n  word  juncture. 
A bit  matrix  of  aliowabte  Overiaos  *5  precompiled  mte  the 
procedur.?,  ana  is  'nde»ed  Oy  ln«  end-  and  begm-p^onemes 
of  the  wo  Cl  naif  Any  overijo  ;ur-Cture  .nvofveng  pnonemes 
which  are  no!  allowed  to  share  segments  'S  rejected  by 
AINCT  In  t^e  separation  case,  as  m iilowcrd  overlaps,  the 


segrnental  ciescr>o'inn  o(  the  spoken  ulterarsce  is  eBemtrsed 
rn  the  rc*r»fcTt  emd-  ano  hetm>-perts^»i>»»  04  the 

pair  to  determine  i*  any  d-ialiOwed  se^'»«nti  are  present  m 
the  juncture  If  su<h  segments  are  found,  the  word  pair  rs 
^ejectea  Only  when  a word  pair  passes  all  ru^  lasts  wh>ch 
aop-y  'n  the  segments,  ccnleit  its  juncture  »s  it  accepted 
as  a va!id  sequence. 

Examples  of  allowable  phoname  overlaps  are  the  (oUowirvg: 

(1)  Allow  words  to  share  a fUp-Uke  segment  tf  urt  of  the 
juncture  phOfs«mes  is  a stop  (2)  A'lo-w  nasal'kxe  segment 
Overlaps  m nasal-stop  phoneme  junctures.  (3)  In  a 
fficatiue-stop  phone'-.e  jurteture.  allow  sharing  of 
aspirations,  fricatives,  s fences,  and  flap’like  sege^nts 

Examples  Of  rvon-aMowed  segments  >n  a juncture  are  tK« 
fctiowmg:  (1)  Oo  no!  a'low  a vowe)  segment  m any  juncture 
(Overlap  or  separa.>c>A),  umess  it  is  a vowef*vowe)  phoneme 
juncture.  (2)  Do  r^t  aiiow  a fricative  segment  tn  any  non- 
fricative  juncture. 

Current  PfSuits 

Stand-aiOise  pnr^crmance  evaluation  runs  were  made  over 
60  utterances  usmg  woros  generated  from  fi?e?  pr(X3L“:*d  by 
the  Hcarcay-II  wcro  hypoihusieer.  Syntactically  adjacent 
pairs  of  words  whose  ratings  were  AO  and  aoove  (or«  a scale 
from  0 to  100)  and  ^hpse  fi'i^rs  (left^word  end  time  and 
rn;nt-wo*’d  begin  time)  we'e  wiitvn  a 200  millisecond 
interval  were  considered  AU  pi  the  wo/ds  used  for  testing 
the  procedure  were  hrpothesizeo  ‘^riPm-up"  *n  Hearsay-li; 
r>o  gramri^atically  based  predictions  were  used  m (Ke 
evaluation  runs  Tab'e  1 summarizes  the  performance  of  the 
JChCT  procedure. 

It  IS  expected  that,  as  lower-ievet  sources  of  knowceTge 
provide  more  accurate  times  fcr  word  hypotheses;  the  -'rtes 
for  acceotshce  of  vand  word  pairs  mav  be  tigntened.  'urther 
lr^creaslng  the  speec  and  performance  of  Hea/say«lL 
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ABSTRACT 

A key  probiftw  lor  speech  urrirrtt^nfijng  lystems  it 
tSc  verl^icatto«^  of  word  hypotheses  ier.e'iJed  by  venous 
Knowledcc^  Purees  >h  the  ir.  paper  we  witi 

discuss  the  generat  prooteni  word  vcnficei'On  m speech 
Uhdefstand<hg  syetems  A oescnption  o*  our  maichmg 
algorithm  for  word  verifiCatOh  wrH<h  tr  nAced  oo  that  used 
io  the  HARPV  sysferrs,  a ijCheraJ  cooneefed  spcect. 
recognition  system  (Lowerre.  1976).  -i  »;.«ven  An  eKjmpie  ot 
the  vcriheahon  Of  a wc'd  oypolhes'S  jt-or  lois  algorithm  is 
presented  Problems  which  arose  m >hp*/«r.g  iru  technir^oe 
to  verificalion  of  >rid»vidua*  words  ‘h  a connected  speech 
understarvdmg  system  arsd  their  soiulicns  arc  o<scussed  A 
performance  analysis  of  the  verifier  n tenvs  of  accuracy 
end  speed  is  g<ven  and  directions  for  tutcra  work  are 
irvdicated. 

INTRODUCriON 

Word  verficaiion  is  the  evaluation  of  word 
hypotheses  *n  speecn  crvJ^i.'St-nd  or  reccgnition  systamt. 
The  aim  of  tms  evaluation  -s  ?c  cscioc  wn  c.*>  hypotr.cscs  are 
worthy  of  further  processing  oy  other  carls  of  the  systenv 
This  evaluation  »s  genera'ty  pertomet:  oy  r.  •asun'^g  hc*w 
closely  a given  word  matches  its  prec•••■•',^v  rccrcsentation. 
The  rcc'cse'irafion  arxl  the  match  of  (f'»  acovsl  c signal  may 
be  performed  at  various  reoresentat  or.,*  'c  t!s  such  as  the 
pararnetftc,  pnor\elic  and  SviUb»c  S.nce  errors  are 
introduced  and  propagated  as  mloratahon  is  encooed  from 
the  parametric  to  the  svJiabic  levri.  jccuraie  maten^ng 
becomes  irKreasingty  O'lhcwtt  at  each  succes$  v«  level  cl 
ibstractiorv  However  the  computation  tfi-?  fer  matching 
decreases  s«rK«  there  are  fewer  match  etemenis  each 
COr>la«ning  more  mfOriial-On 

Words  may  be  hypothesized  from  many  diverse 
sources  of  Knowledge  r\ot  so’ely  oased  uocn  acoushc 
evider'ce.  If  5^  to  37  cf  the  vocaouUry  .s  h>  ooihcsizeo  for 
each  wc'd  position  m the  utterance  il^e  cu*rcn{  h£ARSAY 
bottom-uo  performance),  the  ver:’l'  d‘*tnguiSh 

between  50  'o  60  competing  word  cand.  . tes  in  a ICCO 
word  vocabulary  Even  Uifh  lic^mficanl  improvements  'o 
word  hypo?^  js'zation  (le  di!'.rea*^  r>g  :,ie  effect  ve 
vocabulary  hy  cotnesited  to  5^  per  wcrq  pos  lion).  as  we 
move  to  systen^o  with  large  vccabu  ar^e^  ( '*100.000  *o»ds 
see  Snuth  197 /;•  the  nurnper  of  poi'tiai  vertf.-aoie  words 
remeins  qude  la'ge 

The  verif-rr  must  ass'gn  a like  hood  score  which  is 
commensurate  with  Ine  mAJeh  between  the  uncarlymg 
acoustic  data  and  the  pr^one^.c  descr-oiion  ot  t-?  wo*d  The 
goodrsess  of  a score  may  te  only  ’e'-CO'a'',-  S gn.l-t^nt  so 
the  scores  shuuid  rann  cree'  the  ccmpetihve  words  n any 
time  area  such  Init  the  cCfrccI  •ord  ‘S  nigr  ;n  the  oroenng 


Besides  this  acceptai^ce  entene,  ■(  it  alM  rxecessar^ 
for  thi,  verifier  to  reject  absolutely  • large  percentage  ot 
the  hypothesized  words,  without  refecting  significant 
rsymbers  of  correct  words,  in  order  to  constrain  the 
combinatoric  explosion  of  hypotheses  at  higher  levels. 

The  hearsay  ENVlROhH-*£MT 

Word  verification  is  performed  wtlhin  HEARSAY  I'  in 
the  following  er-'irofsment.  Word  candidates  may  be 
Supplied  from  a ooliCm-up  word  hypolhes'ier  \rCW0Vy) 
based  on  acoustic  irformatir'  m fro-T.  a too-dewn  syntax 
and  semantics  Knowledge  scurce  (SA$S)  based  on  syntactic 
information  and  constraints  prov<ccd  by  the  grammar 
PO^^W  (Smith  1076)  rwoviccs  word  hypotheses  waich  have 
reaionaoie  underiym;  acoustic  support  over  a oe'i.m** 
portion  of  the  utfer?Ke.  The  times  supplied  are  utec  to 
guide  verification  but  do  not  preclude  change.  SASS 
<Hayes-Rolh  1?77)  provioes  word*,  which  can  be 
chjracterued  as  being  syntactically  plaus’ble  m a part'cuiar 
time  area  of  IKe  utterance.  ^-0  pruning  is  peM0*med 
accoremg  lo  iho  credibility  of  tre  underlying  acoustic 
• niormaiion.  S.fKe  hvest  wo^ds  a'c  a-w>ys  hypoircsizefl 
based  on  a previously  verified  word  or  from  the  bouncanes 
Of  the  utterar\ce,  on»y  one  time  n known  This  reau.res  tnal 
the  verifier  must  rvot  only  r^te  the  hypothesis,  but  must  afso 
predict  the  m ssinp  t-mc.  In  addition,  Sines  words  may  be 
preciicted  to  the  left  or  right  of  a verified  word,  the  verifier 
must  have  the  abiMy  to  match  wor.is  in  both  Cirectior\s 

hearsay  operates  under  the  nypotncsize-ano-test 
paraoii^m  to  produce  many  compel. ng  hypotheses  which 
Overlap  in  time.  Each  word  hypothesis  must  be  verif.cd  ar^d 
assigned  a rating  before  it  can  he  used  by  other  sources  of 
Knowteege  Eacn  of  these  verified  hypotheses  can  in  turn 
be  uceo  as  seeds  to  generate  new  sets  of  syntactical. y 
plausible  words  A measure  of  *hc  lan-oui  I'om  eacn  word 
IS  th»  effective  hranching  factor  of  the  HEARSAY  If  grarrmar 
(Ooedman  1976)  wirch  IS  is  oetween  5 and  15  Thus 
regardless  of  the  scoring  performance,  a verifier  must  be 
tompulaliona'iy  efficient  in  order  lo  be  useful  m thit  typo  of 
System. 

VERIflCATlCN  UOOCL 

WIZARD  can  be  decomooied  mfo  three  mi|cr  partj 
word  netwoiKs.  a ser,r,>entat»on  of  tho  gtferanc?,  a~d  a 
control  structure  ’wh  :hi  implements  t.***  ma’chipg  a'-r.bi’  thm 
Eirst,  each  worp  m the  lencon  »s  reoreseniej  by  a sTa'icaMy 
defined  network  wh  :h  empoaios  a'le'nate  oronunciat-ons  ot 
the  word  Each  node  'n  the  wo'd  network  represents  a 
pnone  arsd  arcs  indicate  successor/prececessor  reiauonships 
between  phones. 


This  wc*K  was  cuooorfed  by  the  Defense  Advance  Research  Pro-ects  Agency  under 
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I ^ of  tH«  for  1K» 

word  ABSTRACT.  Thei#  n«l«rC'Ki  at  storeo  <i  a static 
data  structure  'O  a Oax^'ed  ^or'ni!  tkg  un«»orm 
representation  ot  words  by  * S'nr.’e  networfc  whicn 

• mbodios  all  speech  oeoenoer*?  kno-lefl^e  jves  several 
advantages  Ovc’  otKcr  aoproach-es  Tifst.  the  generaiion  o# 
proper  nelworK  oeSf riotion;  can  be  h.iino*ed  On  a case  0> 
case  basis  anlttout  the  need  of  a general  t^eorv  lor  el>  Th<s 
also  el'minales  the  need  lor  spec. at  ca*-e  solutions  wherv  the 
general  theory  fa-i',  cr  is  tourva  *nco**  oicte.  Tools  are  a‘so 
•va'J>ble  to  gene' ate  word  descriptions  and  tune  the 
KOuttic^phonetic  lemoiates  (Lowerre. 

Th#  acoustic  information  ij  a segmentat«or\  of  the 
utterance  where  each  sege^nt  is  represented  as  a vector  of 
phone  probabilities  WIZARQ  be^e  its  from  the  use  of  *he 
sarne  tempj.stes  and  segrrentaiion  as  i^e  ^AftPY  system 
(Lowerre,  137S).  As  »r  mAPPy  (he  pnone  prOOro»t>t  es  are 
dfstance  ‘^eaju'ei  b*»wee**  «*ch  <e<i'*^nr  arv^  -vov't'c- 
phOr\et<c  remplates  m the  Dr*or\et'C  ccNCinary.  T^n  va:-.ie  is 
e scater.  tog  likelihood  meassire  fsirc*  tne  prooab>''f'es  do 
not  aof.i  to  1)  ar^d  it  used  directly  n computing  the  wCrd 
match  ecore  Over  the  grven  $eg*ren(t.  WIZAPO  uses 
epprovimately  90  tempiaies  to  cover  a^l  pfiorset'c  variations 
in  its  1024  word  vocabulary. 

The  las!  component  «S  the  dynir^ic  matching  algorithm 
Although  there  is  no  speech  dopenoent  krwswttro^e  emboO»ed 
in  this  module,  several  heuristics  a'e  empioved  to  f^nd 
optimal  starting  points  ard  to  ciiobse  the  best  fmai  so’^'^enf. 
These  heuristics  are  oiscussed  >n  }h«  following  section  on 
implementation  issues 

Figure  3 illustrates  the  matchirg  of  the  word 
ABST9ACT  to  ten  segment*  of  an  ufterar<ce.  The  match 
score  for  any  phor>«  / m the  Tt/i  seg'*'ent  can  oe  caicuiated 

fr^r.'.  (,iO  following; 

M*  j “ _ I 1 • T 

Wtler*  l^jT_i  11  ll"*  best  ’maten  score  tn  the  previous 

eegment  for  phoHe  J wher« 

J • 1 or 

J precedes  1 m the  net^erk 

arvd  P|  f is  the  aeousttc  maten  score  of  phone  \ m 

eegnient  T 

Figure  2 give*  the  ohphe  probab-iit'os  to.-  t pne-e 
m the  network  ir.  each  c(  th*  sejtments  over  wn-cn  tf>e 
match  IS  performed 

Those  scopes  ir  ^-gurc  3 marked  with  t inp,  »»»- 
best  path  through  the  mipping  The  be$  n time  . . eacr» 
segment  is  given,  along  with  the  segmoni  numoer,  on  tne  i;p 
of  the  figurp.  T »e  left  side  ‘S  laueied  ^.th  each  pnore  -n 
the  network  Fntnes  n the  tab'e  of  a>  nc.cate  that  a pnc'^e 
mapping  to  that  segment  is  no!  a!lo«.cd.  The  f-nai  '*'>rp'ng 
IS  given  at  the  collo*'  at  the  I gu'e  The  iina'  mate''.  $cc»e 
of  40  »S  tho  score  of  the  best  pno-.e  wruc-s  transriio^^  ir.ip 
the  final  state  j ous  the  acoustic  mates  preudO'*:/  o‘  * 
which  »s  oefirxed  to  be  iero  Tm-s  rfo'cspnis  the  !::>fe  of 
the  best  pafh  through  the  network  Ti'is  sec  e «ouid  be 
rrormaii^ed  tby  f^e  number  o*  se^^-c.-'s  ma::c>«c  c • -'ni 
would  receive  a »-£Afi5AV  score  ot  90  po*  of  a pcs'.  - r*  .OC 
Other  palhs  can  be  found  by  trac-n^  bac«  t-'om  tne  other 
four  possible  ending  phones  •-  - ’.«!&),  DX  (b5).  arxj  - 

(46) 

IMPLEMENTATION  ISSUES 

Initially  several  probierri  arose  w^rie  integrating  Ini* 
knowledge  source  •'^to  KtARSAT  ii  The  lo'^owirj  a 
discussion  of  tho  problems  accressed  duri-'q  fre 
implemer^tatior'  Of  the  -eri'icr  First,  s res  we  were  cea-**} 
with  Single  woros  and  aftempfirir  to  et  *^em  as  it  '*'Cf 

• jnsteo  'h  isolation,  many  of  Ihe  constrain:*  crov-ocO  bv 
word  juncture  rgjes  and  synta:;:c  Knowledge  were 
unavaiiobie  for  u'S  In  l<gn!  ot  the  power  that  *h*se 
constrainfs  give  to  S'^’^har  systems  tLowerre  I97b)  wOv>d 

verification  be  •racrioie'’ 

Word*  cOu'O  be  hypofn.es  ;ed  bottom-uo  w.lh 
ir»correct  hmes  ThiS  mearst  irat  procedures  naO  !0  te 
employed  to  search  the  segmental. on  for  the  locai  be^l 


stariiMg  and  enu  n/.  po  nl  arOunu  the  g'ven  pOinU.  WCfvi 
predeted  'op-aown  aiway*  h.i';  a rrtissm*  a-d 

procedures  lo'  pr^^jchri^  these  timjs  accurate-y  h*d  to  h* 
developed  m the  gei>r'-a!  nn  of  ervd  a^d  t<-,  r 

times  of  words  wr-rh  share  vOwrit-  ptlen  cau.'*  'airo  wcifl 
pairs  to  be  rcj-icsed  by  Ngner  levc*  ’nowiedge  sources 

The  conviT'o*.  of  'ntern^  m,»tch  scopes  to  hiAPSAY 
ti  raJings  whilrr  m,j,oiairu?>g  c0ns-5Jer.Cy  Of  she  rjfiri's 
proved  lo  be  a ma  ar  concern  wren  it  was  noticed  that  t*c 
average  infernaf  ral-ng*  tor  worfls  varied  cons‘3craov 
deper»d‘ng  On  where  m the  utterance  ine  words  OtCurrel 
The  nOtiOn  of  r#nK  Order  Of  the  word  wi|h  reSpeCt  fo  IS 
cOmpetilCrs  rather  than  absolute  score  pro*/ed  to  ce 
unimDiemenfabic  i-n  practice.  Whi'-c  -jsetJ  m static  tesis 
out'.'dc  of  the  '.ysJc''^  hr.  a rieasuic  of  performarce,  a rank 
order  scheoxc  wh  .'i  assv.HTiea  all  cOA:c*cliiors  m a paiticu'cr 
time  area  were  av,''*;jr:’e  to  be  rateo  at  the  same  firnc,  pr 
f ememhered  iO';  7 'a*er  proved  intractable 

It  has  been  roticed  uy  other  rcsearc'iers  that  short 
fund  on  wO.'Os  suc.i  as  on,  the,  of,  to.  «n,  tenet  to  be 
hypothesised  at  r-any  places  »n  the  utterance,  with  good 
ratings  and  am  most  crien  false  alarms  Our  erperic'sce  was 
muc-'i  the  same  ana  we  these  to  aftemot  to  hano'e  this 
proticrn  directly  rafher  than  pass  it  on  to  the  high^er  levci 
knowledge  sources. 

SOLUTICriS 

Several  rnoecs  of  veriftcahon  are  supported  withm 
WIZA'^O.  Each  repre^^p},  J parfia’.  solution  lo  on*  or  more 
o*  the  prob'e"".'  ouf’  nea.  fjcn-pad  mpdo  uses  no  Ke‘jr  :t:cs 
to  ccterr.  fh-c  bcondar'cs  of  the  match.  Th-  P'cdccieo 
oeg>n/end  limes  are  mapped  '4'ectiy  into  their  resoectue 
>eg'“er'.;s  and  ver  t*cotion  ■*  prr(orK»ed  nc.O'.s  those 
ri'g-'-r-'s.  It  \^rc\  acp'Or'"*  ite.y  60  r i I'vecc'crs  n{  C*^U 

time  cn  a PD^  kAi"  »;%  inj!^.,  r?  i'*.  rvs  fjn'.d? 

Pad  rnoce  was  added  to  na-o.p  |^e  erpu'em  tn^t 
bollOri>-op  tures  m,;*  be  ir^cOfrccl  This  it*'.  i«  S.'fCnMv 
used  to  verify  a’l  bottCM-vp  kyt»oth<.s;i  r ,ks  mode  the 
bcgfn/end  firr.es  am  trapped  '«>tO  sr,j;*r:,x',  »n  '«!r*-nad 
mode.  Then  3 pr'  segment  vi*<f ' ! " 'v  , .i  *0wp':  2u'''* 
the  malchi.ng.  'ryc  if  3 ts  (he  ivef  n se*me.n!  £ !'ie  e"'0 
segrrr  r.t,  sc;,**. a-:  are  al^Owsi'ie  sti'tmg  pom" 

fc'  the  .'nalcn  a"<J  r.-i/L.  L*i  a^c  tf*  * a'lOTiaofe  end-r^g 

pcinis  The  nine  oaths  between  tt^e  hO;  rriary  segmer'is  a'e 
evaij.’teo  m pa'a  'c’^  hy  rroo’fymg  *rc  oounoa.'y  conouion* 
u The  match'.ng  /c.'.tnm  As  a result  WITARO  rrusf 
cacktrack  from  each  o'  (T.al  er.J  jcjmenls  in  orcer  to 
fir;<j  (he  ;pMtct  t.*i;in  segment  assccated  with  the  path 
This  IS  riecessary  .c  i.'a:  the  beg*n  tinie  cf  ih.e  segment  can 
be  refurneo  as  t»  e oegm  time  gt  tne  wore  a.''.'  to 

fhp  path  'pnfjth  iru''''rr  c!  pncnes  ih^  pr*.h^  for  SCO* 

T n n Cde  !ar.f'  abCut  iOO  miH, seconds  pf  C.^U  tir  in  »ric 
POr*  *'ArJ  i^v  <s  adO.t  j5  !>mcs  faster  than  etpiori*-^  each 
0*  the  r ne  pafn$  in  n.'.n-p.sp  i^ooe 

As  wc  have  m.'rtuj'-ed  be^O-’C  .t  is  necessary  to 
pcrfo'**-  '.cr.'  cdt  h"'  -.  o'e  Oniy  Crj  gt  the  word  tiroes  iS 
K**cw«-t  Two  P'C;:  !•  ' -Cdct  art  *«P'cr.?rleO  in  iMZARO, 
©•ne  where  the  ? -ve  .s  ur.r.ncwri  ..-iiir,;!  *r»o  the  other 

pred  et'  a r><:s  i-g  l-  n time  ’ie’t)  A*  pact  mooe  a one 
segr-cht  wir^Jow  I',  t.a'uated  a'Ound  the  give*'  sf>‘:'h^ 
segment  Then  earn  ' se"''''nf  15  matched  ano  ;h.e 

m^lch  ;:orc  ccmoiilu’  4S  -f  the  vc'e  enamg  ;n  tn*| 

sr.7.-*'cnf  The  scorns  are  ordereo  a-d  ■'•e  score  <cr  •tt 
tvfst  pa!h  IS  returnee  u-o*'7  with  the  m.'inc  limp  Sc-erai 
hour -t I .< s are  usco  ’c  prune  the  nur^rc.'  of  enp 
actu.4iV  ioo^eo  at  posso'c  crv3  ;*a*es  Th  j ■*  me 
COmo'-.f alronaily  e»t^n'.  .e  of  fh®  ve'  f cat'On  wodec  (a'lnr; 
afco:.f  i 60  m-iisecor.ac  per  venficai  0.*  on  a PCP'i^AlO 
process^' 

Several  e«per  ’*en.ts  we'e  perfo'’'r‘eo  to  nete'’'-  nc  'i-e 

best  to  hOr  — 3 ufc  !he  "'a'm  'ne  fee-.-  ove 

emp’oyrd  was  to  ri-'v  apDro»,maie.y  ."'T-TO  bci:o"**'_c. 
wO'd  hyco.hcfe*  ♦*c~'  rO  v‘ ' ?ra»v.' no-mj  .-e  the  S'OfCS 
and  cacyiate  the  a»c''gr  'an«  orse'  o'  !re  corrr''’  wo'cs 
The  rank  orcer  g-ves  the  n.^nt>«r  of  ir-cOrrect  woros  ‘nsf 


ft*  O'  r<«rrr  lK»n,  th#  lOffCCi  *0^0. 

Thts  O^dffiOj;  •'  A mei^'^o'C  “kO.*  man,  wc/ds  PC»  «iOrd 
pOiil'On  ^ni'-t  con^io^rcd  t>y  *^e  Jop  tevei  wftowicr'ge 

tOurcec.  m or.^f'f  *0  I'awe  conlid^'nce  Ih#  CQrrcc\  mcrd  I( 
prcienf,  assuf^-'^E  f Pee^  f^ypoih^^ijed 

t^ve  scopes  by  th<»  tiijfilcn  o‘  !r*e  wofQ  a»«o*‘'‘ecj  Ih# 

probism  of  fuft:f‘On  wOf'^s  fete-v-n^  ui'uiu^Hy  good  sco'’**. 
MO^C  cOf^'P-Px  nOrmaNialiC*'S  oatCd  OA  nOn->»rt<a^ 
tcahr^  were  i»iso  feje<led  ftorma'i^atienj 

employing  pe'^a!t*es  for  'nappi'’^  ta'ne  pbo^e  i*'fo  «4«y 
succosstve  scgmcnls  proved  lo  be  foo  t»me  cooiumtnj  trt 
light  0#  the  ber>el't  ck'fivea  Cu'rr»'!!y.  predict  mode  scores 
•re  normalized  by  th.*  numbe'^  of  lermeoti  ia  the  ma’ch 
ppth  V wt-wle  the  other  modes  #re  r>crma»/«d  Oy  N-I.  'h«se 
normati^plions  are  rompotattonaiy  simptc  aiyi 
embcMtshmcrts  tned  to  date  have  not  performed 
aignificantly  better 

The  convcr^^on  of  mternai  WIZARD  scores  tO 
MEAkSAY  I!  Kyoolhes'S  raiirgs  wes  accorrptithed  by 
corxducfino  a itatistica*  analyst  of  correct/mcorrcct  word 
ratings  over  appr oumaleiy  5COOO  verifications.  ByhAov-mg 
distnfjvjtion  of  correct  and  inco/rect  words  Over  «f»ch  of 
the  inle*na)  score  values  {dynamic  range  of  6*3).  a 

<Of  rcspond»r'g  distribution  Of  h£A*?SAY  scores  was 

catcialaled  The  t^AnSAY  score  o.strt'>wi*on  ai'O-^s  for  the 
absolute  rejection  of  verified  This  threshold  wa*  set 

so  as  to  reject  no  correct  words  Scores  above  that 
Ihrcst'ofd  were  a»5t.'»buted  so  as  to  capita'i^cd  on  the 
distribufiOr\s  0^  the  correct  words  Severat  tradeoffs  ire 
possible  here  If  one  .'ecu*res  that  no  potential  cpr^^cct 
words  be  receded  then  WtZAs'O  abie  to  reject  124  to 
107  of  the  irvorrect  words  nypoihes*2eo  On  the  olher 
har*d  if  it  were  pos'iible  for  the  system  to  perform  sv»th  a 
small  number  of  the  correct  woi as  De-ng  rejected,  a h 
percentage  of  nKOrrecI  words  cou'd  ce  'ejected.  A^iow  ng  a 
67  rejectkCn  rate  of  correct  words  aopfCti-'ate*/  517  of  the 
irsco/rcct  words  can  b«  elimiratea  from  consiceration  oy  the 
higher  level  ^rsowieage  sources. 

To  a»d  in  compentating  for  the  apparent  temporal 
d'ffcrcr'ee  m word  scores,  the  acoustic  match  prooac>»'l«« 
gerserated  by  the  se‘3mei'.fer  were  rormaiueo  such  irjt 
score  of  the  best  phone  «n  a sesment  nae  the  absolute  cesl 
match  score.  Th^s  aiievfated  the  prob«cm  ar»d  improved  *he 
reltabiiity  of  the  norn^aiized  match  score  wh:ie  ieav.ng  the 
rank  order  statistics  unchanged. 

RCSULTS 

The  results  sunrnariied  m Figure  4 are  for  fi/e  da'a 
sets,  COnfainmg  100  utterances,  m wh'Cn  332  correct  words 
were  hypolhesized  bottom -ud  bv  P'rMOW  in  addition, 
13053  ifKOrrcc!  words  were  ^eneiafeo  T^e  vocaOuUry 
z-if!  for  rX)M0«'t  and  WIcAhJ  was  aor*rc«ifnateiv  550  woros 
WIZARD  rated  each  of  the  words  usm;  pao  mode 
verification,  for  each  rating  fhresho'o  U5,10)  the  ry—oer 
of  correct  ard  uxorfccf  wO'ds  lhal  were  acceoteo  or 
rejected  ;s  tabulated  From  tnis  cata  tr^  nur-Par  of  words 
hypotr^esized  oer  word  oo»i:  on.  and  the  percent  of  the 
vocabulary  per  werd  position,  ran  oe  caicu-aled  These 
numbers  g've  a vocabulary  •r^-penpenr  measure  of 
performance,  a!.ow*nn  compjr  scris  oef’*een  var»oos  system 
configurations  An  average  ran^  oreer  of  the  correci  woro  IS 
provided  which  measures.  ai  each  tnresnoid,  the  number  of 
words  in  each  word  pos-t'On  inat  must  be  examined  m order 
to  irxiude  the  correct  word  The  range  of  r^rn  orsers 
between  the  data  sets  (2C  utter arKes /set)  is  also  noted 

DlSCUSStCt^ 

The  major  d'rccf*0n  ©f  fn-s  wCrK  .$  the  aopf«Cal'On  Of 
fh*  HARPY  nelworK  rcpresentaficns  fo  ine  ver  r<cai'Cn  of 
Single  words  in  a connected  soeecn  ur'oerstancr'g  system 
This  incii>des  the  rnoa.ficat-ons  *0  a''0w  the  var-cus 

verification  modes  cJictated  ov  i**e  ^ARSAY  II  s.stem 
strategies  We  feri  thal  ‘.VlTTrO  m^Kes  an  impo’tanl 
contribution  to  th»  ovc'al’  cer  iC' mane e of  ^tARSAY  it  an© 
forms  a grounowor*  -joan  which  mo'e  sopn.shcated  >erif  *rs 
car'  be  d^velopert 


Seve'a!  problems  with  the  current  word  ve»'»''.  aion 
System  can  not  be  solved  wiih»n  inc  egisfing  framework. 
Fulure  worK  IS  rcG-'i-ed  in  the  fo-'Owir^g  arjav  •<■*/ 
sefsemes  for  r* /ahon  of  scorC'.  ha-e  been  prccosed  lo 
improve  Iho  per  fo»  a ic e m se'^.'  cnlitioni  havirg  many 
very  slvort  transition  vngmenft  *l’••ve  ieg''i'*nts  m r.cnerai 
have  pujr  ratmgs  aruj  often  degraou  the  cOmpOS'te  word 
score. 

Although  we  felt  thal  the  matching  a^orithm  was 
COmpi.t  atiOnaiiy  ctf'Cien}  when  first  impiffmenled.  as  system 
strategies  evolved  it  was  found  that  a s>gniif:canl  portion  of 
recognition  time  was  bemg  spent  m verification.  A s-rab'-e 
irvcrease  m speed  can  be  oolaired  py  cooirig  certain  of  *Ke 
irwver  loops  :n  assembly  ianguiC^*  Other  icmenfalion 
oriented  optimisations  may  be  nee-"nd 

A most  useful  addition  to  v^iJAi'O  would  be  the  ab*‘ity 
to  verify  seouervees  Ct  words  by  o.-namic  generation  of 
muttiple  word  rvetwo/t-s  These  wouid  empoov  tr\e 

appropriate  word  juncture  ru!es  an.a  wovd  atiOw  WViTAnO  lo 
rate  phrasal  hypotheses  directly  rather  than  having  ct^'er 
knowledge  sources  calculate  a cor'posMe  score  from  th® 
individual  word  scores.  Along  these  fines,  perhaps  as  a first 
step,  it  IS  nei'essary  to  handle  word  juncture  proo'cms 
which  cannot  be  stal'cally  erveoded  in  the  sir'gle  wpro 
fsetwofYs  Ihemcelves  "^hese  juncture  problems  are  a major 
cause  of  incorrect  fir*ics  on  »crd  hvpciheses 

It  will  be  rvecessary  fo  augment  thi^  word  ver-ficafiOn 
Sys*.en\  with  a com.po^.e•^^  to  perform  rrore  direct  Signal 
matching.  The  purpose  of  this  aodition  is  lo  d*^ '■.ujte 
competing  words  which  have  500c  VVIZAkQ  scores  -n  Ine 
same  time  area  V^c  propose  to  extract  word  templates  at 
the  parametric  fevci  and  per'or*  matching  using  UjRura‘s 
method  (ItaKura,  }9"5)  The  philosophy  here  is  Ic  store 
ternplates  for  a srna>i  number  of  pctenliaUf'  d'ffKuit  wpros 
ral^.er  than  synthesize  the  templates  by  a ruie-oased 
system.  This  time  consuming  matching  would  be  penormco 
when  indicated  by  h,^ner  sources  or  AJiowieege 
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Tre  d’  of  Sig^  Cetection  Apoxed  to  Speech  S&gmeniatiah 


Hervy  G.  Goiooefg 


^ep3itm«r»t  of  Compulor  Science 
Caf-r^g  e MeMon  University,  PittsCi^'ih.  Pi  i5i:l 


Abgtf j<t  stJ?»sts£i)  a',  from  Signal 

Cetecfton  Theory,  beer*  mown  to  pa-'anNeti-re 

iKe  “6elecl4*>iii« /*  of  Signal  cvcr  «n  a wt'ie  var>*ty  of 

perceptual  S'fual^orij  Its  use^u.«^si  i%  e*Jer*a<*d  >0  t^^e 

problem  of  Oui!'tify‘''S  e^ror  *ates  5egrrer»tjt»0ft  c* 

confmuO\iS  speech-  !(  Ni$  often  ceen  inr.possio^ 

accurately  compare  O'He^enf  '^ach^  ‘echn^Ques  ♦or 

segmenTatiOn  sirxe  errors  occur  iS  Cither  missing  Of  6/‘'a 

segment  oocr^af<es  wlK'se  rates  ar®  reialeo  by  nte'.'iai 
Oecision  thresnoIcK  The  t)as*c  d'  'podel  *s  shewn  tc 
accurately  (>96-  confidence)  Ocscrioe  ?n«  mitsng  vortus 
extra  segment  trade-Off  found  m at  lea-J  One,  '^on-tr 'vtai, 
speech  segmentation  program..  (GetTo) 

tntroou.fion  The  last  teu/  y:A“:  o*  corrputc'  soeec-" 
recognit.on  research  ha^/e  produf^a.  awong  cmer  fh»-is,  a 

numQcr  of  teclvnqi>es  for  ma;n«n«  segmenitotion  ot  tnc 
speech  signal  »n;o  phcnet.c  lor  acoustic)  uruis.  (eg  C.'7^. 
6ah75,  Go‘75)  Th^  C»ft*cu-‘t'CS  n,o./eb  m ev?tuat;*'g  ano 
compar.n^*  the  periormanse  ct  segmentc's  seem  ‘o  cccj*  n 
two  areas  one  must  ac^u.re  » 0€*.n?l*on  C-  'r.t 

’correct'*  segrnents  ‘or  some  Urge  sel  of  cata  Th.;  -s 
usually  Oor>e  by  hand,  aiinougn  some  iu*on»auc  tetnn  ques 
a'c  a;ai'able*  S'n<e  the  prodNAt-on  of  ’cO’r.rct’ 
segi^cntahons  ano  tneir  con^oafsen  witr  f^.»c-xf>e 

segmentations  <eg.  VVhaf  amovnt  ot  m s a^i » rimer t etc. 
snojid  orve  allow’)  mvoive  a numpcr  of  ngu  ;•.<  as  wc  «.*s 
recognition  system-specif*:  ssues.  we  ..v-  no*,  cea: 
wtn  these  probipmc  vre. 

>-*cwever,  second  p'Oe-e*^  >%  •eg’^en'iTon 

er'-ors  occur  in  !w0  t)  PCS  n-.Ssed  bO>'noa'-es  i*eg'“cn:s' 
a-r'  rilra  oou'ioj'ies  'segv.e»*rs  T»'e.'e  s C‘e»'-v  » trane- 
Qif  between  these  two  V.ces  o*  ^r'i\  pg!  we  ^ar?  of 
uncierstoex*  if  we!f  enough  'i*.  j cuc •ta‘i'. c se'^se  ‘o 

CDrr.pjr.'*  diffcrcr*  segmcnTC'S  IC'  "ven  th?  San*-e 

secmenier  “Tuneo"  to  a cjiffe*'C'’f  pent  c:  -•••e  irace- 
erf,’-  Wna*  was  f^eoco  was  a ^ocn'  o'  *r»i  Irade*ctf  wr..t''. 
yeioed  a smgie.  coniparapie  n>cas-.^»e  o*  segment.*. on 
efficacy  tc^  any  set  c:  data  marted  rr.ss«rg  or 

e^tra  5*-ch  a modqi  -s  provccr:  by  S'gna*  Cetei’-Crn 

Th-eory  wVc  will  shOw  that  the  ’^v'O'v  njte  *c*' 

-w  in  {he  resuiis  of  a 5el  -•'  se^n'e.*>rai'b«  tr.as  r„A  to 
e*p'orc  th.'  M/t  trade'Ofr 


S Rngi  [Tc * e I . I s r ~heoi~y  theory  of  i*gnai 

C^iievt'O-,  as  for^_*r.»leo  Tanrer,  Sarets,  arvo  Greerv, 
t?a'i6A.  ,.!c64]  ts  O'  -'-.,-  i-y  appwod  lo  deJ.'cteon  tnais  v*nrch 
mav  toe  co  -s  derre  !o  the  segmenfatior  process  A 

detection  tr*/!  o'C'.en^s  a stimuli/^.  wtv;n  iray  be  composed 
O*  noist  or  '-f  r>n  jc  ar.<;  cOw?  kr.own  srgn^),  ^nd  reQiwre*  * 
•icc-S'C-'s  td  i.“  ato;i^‘  Ine  presersce  of  the  sigrul  Th^s 

♦ S rcit  ‘jni  Vp  the  -Jr:  siCn  prj<ess  resi.'U.ng  »n  ‘^e 
;iu:entc''}  or  a ^egr^er-t  bcu-'’<dar/  based  upon  local 
ir.tcr  .••*.pr\  ori|-  !{  s asiurred  that  the  » pr«or<  lu^el.'spoas 
anr:  rests  o»  .•a'^*LH.s  er^-ors  are  to  a decision 

proce-is  whi>c--.  series  . -vf  pc.isioty  Irarvsforrws  the  sln»*Uus 
into  sor.  C'  ‘Ai-rna-  sif,nai  spa.e  before  if  yields  a toecs.o-. 
on  :cyc  C'^'seixe  ot  lr<<  ■>  f.nal  The  0«fecfC>'*s  sensory  zs-i 
IS  r C'**.,  *n  Ir-S  n-.CP2l,  IP  be  to  a 4‘n^'.e 

decs. or  pa'ame'.iif  Ai>  cptimai  or>e.  according  *c  Oec  siOn 
theory  t:\s  rai.p  c>(  ine  probaoi^v>es  ot  two  hypoltseses 
'•  the  input  stimuUs  was  signal  p^us  none  or  that  d 
wao  rnptse  fVu*e  A i r^pi.^  ‘h-reshOid  On  th;5  smgft 
paramc^ta.  rr>a/  c«  placed  Ip  col  tre  espeefed  costs 
given  a pricri  ».n.e  .noo.:c,  costs  Of  nusses.  tai^e  a»armr  etc 
^'guriy  1 represenfs  SvCh  a h,potheiicaJ  interrai  rccis*on 
parameter,  C 


( ( 


rlgu-e  1 r -na.  Ceteclion  {.<idei 


*The  ►i;»rpy  spcccn  e c cgnif .Or  systeir  !l.ow/6]  :an  c« 
fo'crd  *0  !rie  cc'rcci  woros  Tr.s  0'i.oucet  a 'best'  ^ • cf 
‘•"C  SfSten-’s  acoustic  and  pro'‘0}bg.:a»  mcw'ed^?  -O  :ne 
s ;ir  a VV'ien  a very  f.ne  f't  »s  mace  .a.arage 

5:o..'.{i:  scg“i'r“r  oO  * ;,  t**c  'tc— * -g  c 

se'.'^  cnis  Cf  y c cse  'O  -nc-e  preoccen  oy  humane 


Ve'y  -mpi,  s'ateo.  f~oce»  assumes  a 

cec  s*on  cjrjT*ntr'.  c when  ar.^  sensu^, 

meaiL.' c-r  enf  ere  w.s‘ f",  Tue  e >’.f  ibuiiO'*  cf  c v.s'i.es  ‘o' 

t >'r  'w  .n  lyres  b*  ■ 1 'a  • O;  wS  •'^Oise  1‘0  "Oiie'^orvc. 

a‘i?  lO  je  r-»  — w-fn  ec-i  •■»'  a-v:e  .<>  r- c 

S mp-c-sf  .,'ir.n  C'  •‘“r  -jfj'  by  n’ 

t''~os  "'e  : .rr  a'-cn  Z'  ‘i  •;*  "fr-’.e 


Reprinted  fro:-! 
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. Pf ; JiCCCpIjtignaU  an<J  :accepl|nOi&e] 
.'espcct'^.trfy  --  arfr  loHicjent  lo  jeiefnune  t«‘»8  *<iast  rt'  for 
wh»ch  jfA  noJimai  oeciS'on  process  can  Otspiay  observed 
rates.  ^h(>n  the  hi(  and  Ulst  csrm  rates  are  plotted 
ip,5»n*.t  o/'e  9r.o\t'.c'  for  a number  Of  sets  of  t/«a;s  wh^rt 
tne  Oatector's  acceptance  Ihreshoid  rtas  o«en  alferec;.  a 
■ esponse  opn'alor  characteristic  ^C)  curve  <s  ootamed 
(sen  figure  i). 


0 ^ lb 

Pr{accep!  | no'sej 


figure  2 Typical  PCC  Plot 

The  theory  stales  that  tre  curve  is  tctaMy 
oet*?r^»ir\ed  by  A'Jien  the  a*es  c*  fhe  ROC  curve  arc 
li  jnsfor'neo  ty  *ho  in-  ersc  funef  on  of  tre  i\crmal 
dist*‘b‘jtion  lunct  on.  the  curve  a apprOKimatcly  a straight 
l•••n  with  s:r'ce«sigma(no:ce)/sigma(signai>  and  x- 
in'orccpt  •d*.  (£g.»6*l] 

This  theo»y  ha-:  oeen  most  oflon  aopiied  fc  detection 
tr  aic  to  provide  osiimates  of  the  detecuo  '<1/  of  the  s-^r^ai 
as  f aapearo  m a human  perceiver's  -nternai  sensory 
S'p.nai  space.  The  >st  mat#  of  o'  provided  by  the  s gna; 
aetect'On  frpdei  mjy  hen  be  cornpa'ect  w.;fr'  weii  n-rown 
properties  of  vcs^ai  or  aud'torv  signals  to  provide  a bound 
on  :he  efficacy  ot  the  perceiver’s  transdccion  process  -- 
the  senyory  channel  VV^i'a  t^e  m«nn  tnrosl  Ot  its 
appi  fal-on  -s  no!  '■ele  -.inf  here,  fhe  s gral  detect:on  moaei 
dnci  the  dimensionless  measure  o'  can  be  used  as  a 
nofh  allured  rneasurn  of  segment  bounoa'y  cJetecfion  Inal  is 
rc’^-tiveiy  unaffected  b>  ad;usfr^(jr'fs  ir  the  p-oporticn  of 
rriissirg  -ersjs  extra  segment  er’crs  P’^rihcrmorc,  In#  d' 
\a!ue,  estimated,  may  be  used  to  predict  tne  entire 

response -ooerato.'  characteristic 

Se.-irrepta*‘Cn  The  resuile  reported  here  are,  for  the 
n -ys!  pa-'f,  obtained  f.'om  a segrr.cn‘ai-on  program  written 
lOr  ? '.'ip.Trrson  Study  of  pa'ametr  ; rep'esentat  ens 
(Gcl’bj  anc  used  'or  a wh.ic  ac  the  l’'•5.a•  s-gna'-to-symool 
slage  cf  the  Mcarr^y  ij  speech  unoersiancing  system. 
[fr-'iTd]  A snort  description  of  ihe  s..'gmen.rer  i$  therefore 
ca -cd  for 

"he  s p.’-a*  anip;  f jdr.  measi-res  ©♦  s-gr^a'  anp  of 
ar-piiluae  cnang?-,**  '.each  measi.-cd  over  botr.  \0  ar-.c  3C 

**5-gr  ji  change  *:  'yp^coJ’y  a p^tte'n  recogr.'-on  maicn 
score 


m;:;  intervals),  a^o  mout.  Speech  n separated  Iror.i  silence 
and  from  near -c'ence.  and  flaps  are  d-'lected  by  their 
anipi  tude  centours  Then  the  measures  Ot  change  are 
inspected  for  s-gnificant  pea^s  (possible  bOuixJaries)  The 
union  of  ail  such  detections  is  processed  by  • correction 
routine  to  merge  muitipiu  boundaries  caused  by  the  same 
underlying  phonet-c  change.  The  program  has  two 
advantages  for  this  study.  First,  the  input  para/metne 
representation  is  easily  changed,  arsd  second,  the  miernai, 
segrnent  detection  process  <5  easily  tuned  along  the  M/£ 
trade  m3ff 

Pesuits  from  this  program  were  compared  with  a 
"corrected"  hano  segmentation.  That  iS,  the  mKhine 
segmentation  was  compared  to  a phcnemic-ievel  human 
segmentation  for  discovering  missing  segments,  and  to  a 
finpr-gr^in^d  phcnetic-ievel  segmentation  for  discovering 
extra  segn-ver.t  errors. 

ffes jits  The  first  experiment  validates  Ihe  Signal 
Detection  n-oce>  rssuripticn  of  two  (nearly)  norma* 
distributions  in  a Signal,  hypothetical  decision  variable.  A 
set  of  ^0  ser'tences  *'-*h  1093  phonemes  and  1541 
pnc.nelic  segments  w?z  <egmented  seven  times  Internal 
tfireshoids  were  vaivd  to  produze  segmentations 
perfornvng  over  a woe  range  Of  the  W/£  trode-O^f.  The 
resu'*ant  error  ratei  a-'o  plclteo  on  a ncr.'^ial-normal  gnd 
in  P'^gurc  3 A 'east  - c.Quares  regression  fit  a l.ne  with 
Slope*!  00  f^JOlse  staiv.ard  deviation  / Signal  standard 
devutiori),  and  t •.niercepl*2  25  (d’  --  the  separation  of 
the  means  of  the  two  distributions). 


P'  (accept  i noisc] 
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Figure  3 T •.  ► Trade-off 

The  line  IS  the  POC  of  tne  segmer.fsr  with  this 
parficula’’  oi'ar-ntr.c  representation,  “correct*  segmer^t 
definiliors,  etc  fur  an  M/t  traoe-ofl  tuning. 

A sec9f“  »r  -**.ent,  run  with  different  input 

oarameters,  «.  pcasurc  of  confidence  -n  Ihe  o' 

estimates  When  -^3  sentence  were  divipeo  -nfo  10 
group::,,  ana  esnmates  of  d'  maae  lor  each  group,  the  351 
cor'f.de''fe  m'e'v.--  i^'  o'  was  fourid  to  be  *•  0 14  Jh? 
estimate  iron-  scr'tences  Ms  the  o’  corrputco  from  ail  40 
wiJh.n  ihp  C'nl.denc?  -^tervai).  Snee  this  "n'ervai  s 
consiorr a;^.  , sfr'a^c  : -an  t'-e  df'e'^er'ces  found  ceiween 

S Jg'^r  nj,-.*,pr  0"' y O’  between  n:j.^  l pa'amcrir-c 


1 

I 


i 


1 /. 


rcpresenf ailOf'*-  wc-  soch  co^piJ^-sons  are  Mean-rsgJt-? 
USiHg  a*  for  ena'»-pie,  Jour  r cpresen*aNon  of  the  sgral 
were  ic-/fed  (Go- ‘5.  /.do  ng  u >.  aiues  irom  I to  2.38 
FurJhcrmoru,  OijO'.si.i'ci  ri^u*!s  Of  t*o  otoer  seg'nentgfi 
[Bnh7o.  C>k'’5'!  al  .i*rd  estimates  o*  ic  oe  ft^aoe  of  2 26 
anc)  2 73  Th^  orcier:nig  Of  a.;  t^eie  segmenfation  •■'unt 
agrees  vc^v  weil  with  our  intuit'Oos  jooct  the  programs,  as 
well  ac  With  tr^  (somewhat  sparse)  results  of  speech 
recognition  use  ot  them 

Conclui»'ons  We  be*-eve  )."i8t  the  rrpoel  prouicco  by 
Signal  Detection  Tt\eory,  and  pa»ticu.d'.y  the  d*  parameter 
of  that  model,  offer  a highly  suitable  and  attractive 
meaiurB  of  T-egmentahor.  eff'cacv.  and  a means  Of  belter 
jnders  1 andiig  the  W/t  lrade*off  Different  segmenfers, 
COnfornuog  to  needs  Of  different  speech  recogrilion 
systems,  can  be  quantilatn/ely  compared,  ana  their 
pertornAance  under  oifferent  'tuning*  of  the  M/E  traoe-off 
can  be  prcoicled. 
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abstract 

This  paper  i jfnrr.ari^es  iniJiat  oeve^Op/rfot  0^  i System 
for  visual  and  -.erbai  data  aca^'&'t  on  in  fhe  cartogr^pny 
task.  Visual  irtpjl  and  output  i prov.ded  Dy  a graphics 
tablet  in  conjunction  with  a grapr-c  Oiip  ay  lerir  nal.  Verbal 
input  cons'its  ct  sequences  Of  commands  and  map  feature 
oescripto's  vuntch  are  recogn  red  by  the  ^«arpy  speecn 
recognition  system  An  important  and  intereslmg  aspect  of 
this  researcli  <nvoives  "s  design  and  analysis  of 
vocabularies  and  grammars  ’O'  tasks  ot  this  nature. 


ir.TgQClCTlGN 

The  cartography  lasK  is  an  mfe-csting  application  m 
nian-mach.r^e  COr-iiun:catiOn  cor-0'n.'*g  se*era!  forms  of 
input.  It  IS  a practical  task,  used  o.)'--  0/  map  makers,  and 
has  a wicii  downed  protocol  in  tri>s  task  features  are 
sciceted  and  traced  from  a rrao  arxa  further  described  oy  a 
sc*Quence  ot  descriptor  phrases  The  grapn.cal  mput  »s 
oOtair«ed  using  an  i-y  coordinate  'npul  cevrce.  sucn  as  * 
fc,raohics  faoict  ir>  currently  oSCO  ca-tegraohy  Systems,  the 
te’rlu.s!  cicsc-'ptions  arc  entered  vu  •eyboard  This  paoe' 
cJcscr  L'Cs  the  Vies  system,  a cartcgrdcny  system  m w^.ch 
connected  speech  mpul  replaces  keyooard  mpgt.  VIC3 
s:andc  'cr  Vo-cc  inout  Csrtog-aony  System 

Tn.s  projcc*  was  undertaken  oeca^se  -t  represented  a 
p'-artical  an<i  uSC''..'  app'‘Cation  Ic'  speech  nput  of  suH  cient 
s to  be  inlcrcslmg.  .(jI  s^’au  I’ncug.n  'o  oe  feaS'O'O  An 
irooriant  aspect  Of  t'^e  research  »s  *he  Pursuit  of  a 
n’elhodoiOtiy  for  language  aes-g"  tC'  r-ah-macnire  vo-cc 
c o~iniijnic aticn  Interact  ?n  w«!h  the  user  15  su*ticentiy 
flexible  to  aiio-v  the  nvest^'a*  cn  of  several  0'f‘ercnt 
n-elhodc  of  language  i.ruCtuT,  tr  i-n  • tf.>  or  no  onslra-n!  to 
highly  consi'amcd  son-,  ences  further.  ' '♦ce  a smpoth'y 
inferact-ng  sysit"'  *1?.-  adequate  rc^.po-.se  wou^d  nave 
ir-imed  ale  aop'  caiiOn,  tncre  'S  f/cat  potent. a*  fC'^  study  of 
the  many  problerr^  assocatr’d  with  man-machine  systems. 


“r.<  work  Supported  n ca'!  c «•  'ne  Cefense 

A ;..»nceci  Pf*',ear-''.  r*ro,ec!s  ~5ency  •FC4ocC'T3"w- 
OOTA)  and  s '-onilorec  oy  tne  A='  Fp'ce  C'f'cc  o‘ 
Sr  eni I f.c  he  f’a'’:" 


In  O'-der  to  combing  voita  and  graphical  mpul  m a 
practical  system,  one  needs  1)  a speech  recognition  system 
capable  of  recogninng  utterarsces  from  a language  as 
complex  as  required  fcy  the  task,  2)  a graphic  system 
suff.cientiy  flexible  to  a';ow  graphical  input  arvd  visual 
feedback  as  necessary  lor  the  task,  and  3)  tome  method  Of 
interfacing  them  so  the  system  behisres  m a way  which 
appears  as  natural  as  possible  to  the  user.  Two  systems 
designed  at  Carnegic-Wellon  University  provide  the 
necessary  tools  The  Harpy  sc>eech  recognition  system 
[LOwc/fc.  1976  ar»d  1977]  recognizes  live  vo»ce  input  with 
the  ability  to  apply  J,r^n••.matlcai  constraints.  The  SPACS 
grapnic  system  (Greer.  i9"6j.  Originally  built  as  a stand 
alone  inter  active  grapnics  editor,  uses  e tablet  input  device 
m conjunction  with  a graphics  display  temuni*  Us 
caoabilit’es  include  free-hand  line  drawing  and  the  ability  to 
create  tables,  fJOw  c.harls,  logic  diagrams,  and  other 
schematic  oiagrams  The  interfacing  probiem  is  solved  by 
the  use  of  a tasn  module  n the  Harpy  system. 

Other  syslerts  fo'  cpeech  input  are  available.  The 
isoiated  word  reecs'^'t'O'^  system  developed  by  Threshold 
TrcnnolOgy  [Ma-'tin,  J975]  ard  the  BcH  Labs  conrweted 
speech  system  (San-.bur  and  Rabiner,  1976)  ara  accurate 
systems,  but  at  present  lack  the  desired  fienbii-ty  m 
structuring  the  g'-anmar  Other  successful  systems,  such  as 
Hoa*f  ay-ll  [Crman  et  si . 1975  and  Lesser  el  al.,  1975),  HWIM 
(Woods,  1976],  and  the  i0M  system  [JelineK,  el.al.,  1975  and 
Bu-ni,  et  al . 1976,1,  ha.  e much  more  e'aborafe  control 

structures  and  were  des-gneo  for  larger  tasks.  The 
Overhead  m.-blvrd  i.n  these  systems  is  considered 
unacceptable  to*  lasKs  s-ucn  as  this  one. 

l*±  "At>pv  cc.'  :'.f  r 'r  r ifiiCj:  t^ECCGMiT:^^  s^'STey 

Ir  the  >~A'Oy  system  the  recognition  process  cors.sts 
o*  seai-chm’  tor  the  b»'*-t  path  through  a precomoiied 
nclworK,  gwgn  the  acuust  c evidcncc  present  in  the 
uite'a.nce  sea.-cii  s'.'ieme  uses  heur  st'CS  to  reduce  the 

nun  bnr  ot  oaths  ; c-nsioc'ccj.  resulting  i.n  on,y  a few  *best 
path?  oe'^'g  searci^ed  m pa'atiei  The  recognized  utterance 


■^he  authors  w sh  !o  acknowledge  '^ai  k'eddy  ic'- 
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•v  thAn  turned  cv'yr  to  i tasK  *nodui«,  9 program  who<e 
purpose  IS  to  resporvd  to  the  user  in  » way  appropriate  to 
the  The  Siniplest  tas^  moduie  wouSd  simpty  type  the 

recognised  utterance  cn  some  Output  device  such  « a CRT. 
\n  more  complicated  cases,  S'uch  as  the  At  abstract  retrieval 
taoK,  the  teoK  module  would  extract  the  iritent  (mea^og)  ot 
the  utterance.'  consul!  >t$  data  base,  and  supply  an 
appropriate  response,  eg  There  are  17  articles  on  that 
topic" 

The  recognition  process  in  Harpy  uses  a precompiled 
network  which  integrates  syntactic,  lexical,  arsd  word 
juncture  Kr>owtedge.  Syntactic  Krsowtedge  is  specified  by  a 
coritext -tree  grammar  defining  tne  input  >ar>guage  Lexic^ 
information  is  err.bcdied  tn  a symbolic  phonetic  dictionary 
containing  pronunciations  and  alternate  pronunciations  for 
each  word  m the  task  language-  Word  juncture  phenomena 
are  character-zed  by  a set  of  o priori  juncture  ruies  giving 
alternate  pronunciations  of  word  oeginnmgs  and  endmgs 
based  on  the  context  cf  adjacent  words  Atl  tr-iese  sources 
of  Knowledge  serve  as  .nputs  to  a program  which  compiles  a 
network  represent;ftg  ajj  poss'Cio  pronunciations  of  all 
possible  input  utterances. 

The  acojst.c  evidence  used  to  cetermme  tne  best 
path  m the  netwo^K  ,s  obta*ried  bv  legfientmg  the  input  and 
extracting  lFC  parameters  for  each  segment  These  LPC 
parame-v'S  are  maictied  with  phone  templates  to  produce  a 
metric  oelwcen  tf  e sejments  an<j  tne  symoO's  <phones^ 
iissocioted  with  network  states  This  c ;s  m the  form  of 
the  probabiity  that  tro  segment  s .-.n  instance  of  the 
symQol.  Probabilities  are  learneo  from  exemplars  taken  as 
tramms  data 

Creating  a r\ew  task  for  h4i'.->y  cor^slsts  of  de^r'lr^g  the 
language,  framing  fhc  phorie  ten^oiates,  and  spec»fymg 
task  rnoduiu  To  define  the  language  one  first  specifies  the 
r.ronmiar  lor  'he  mput  language  anc  then  obiams  from  a a 
t St  o'  a'l  the  woi-ds  used  m the  language.  Fcr  each  of  tnese 
words  a description  of  'Is  allowed  pronunciations  i$  entered 
into  the  dictionary.  These  desc'  plions  are  m terms  of  a 
standard  sot  of  pnones 

IhL  V1C5  SVSTuM 

The  task  modu‘e  coordinates  verpa-  and  g'apmca' 

input  a.;d  controls  aiC'iOjrse  with  t^c  user.  Figure  « snows 

a user  a!  the  sraph.es  dispioy  •r.teractir’g  wdh  the  VIC3 
syf-tem.  Vc'-Qc*  -'nut  s a sequence  of  words  O'  pnrases 
wh  en  may  bC  CO'^mands  'C'  the  lasK  mocule  or  oescrip’.crs 
Of  the  n-.ap  feature  G'.Tph-c  mput  is  vm  a craphics  laolct  *- 
y 'ensor  There  .''C  'wO  graphic  .rput  -OCeS'  pO  n{  mode 
ano  trace  rrode  The  user  en'c't;  pent  rrpne  b/  sayng 
■■pO'nl"  or  “pomt  mode"  i-t  th  s r-oce  the  user  defines  One 
position  on  the  nap  corrcspcnoi-g  to  the  location  o*  an 
featv.re  such  as  a we'!,  pono,  Cr  wa:e'  tan^  For  r-orc 
cpn.piicoicd  and  'org’r  'ealgres,  such  as  lakes,  islarujs, 
shorelines,  ana  ha'bor-L,  trace  r'.oce  & entered  in  ;h«s  mcce 
the  x-y  sensor  pDs.I.on  15  cont.n^pu-s-y  monilorec  g vmj  a 
grxpnic.'il  nc'cr.c  lOn  cOrS'Sl-ng  o*  a S"?  pt  ’ nes  in  ooth 
r- odes  the  praphicai  ccsc'’io!'On  s 0 s?  ayec  on  a CRT  tor 
- uerif'caiion.  F-gu'c  2 s'^ows  ho*  me  g^apnics  diSp'ay 

appears  afte'  the  user  has  'raced  an  .nler’-ittent  sirean. 
At  iho  DO  nt  the  use'  oes*' hes  me  ♦eatu'e  veroa-iy 
accoro.ng  10  the  ••ocaouiary  and  ?r snir-a'  cai  structure  The 


Figure  1 c.'ser  at  graph:c5  cfisplay  interacting 
with  the  Vies  system. 


F.gure  2 Gr.iphics  Ci:ptay  after  tracing 
ar\  inte'n-.  ttenl  stream. 


Figuri:-  3 Grapn  cs  dispiay  after  verbaliy 
aescr-D  ng  !'-e  intermittent  stream. 


d'Splqy  after  verba 'y  aesenomg  the  stream  is  shown  in 
tigu'C  3.  Figv'C  ‘t  shows  the  display  a''er  another  t'acc- 
desc-’oe  cycle  ccsc-b-ng  an  adjacent  por'.d.  After  the 
desc''’p!iOn  -s  ;orrp  ete  !rp  user  may  rc.ect  Of  accept  't 
vo'cc  commard:  accepted,  the  o€sc''ption  is  stored 

*0'  futu'O  use 


’11.7 


1 


Tigars  4.  Graphics  disp<d>  alter  deicripl^n 
ot  both  stream  aod  pond 


figure  5.  Exemplos  from  cartography  feature  charts. 


The  vocab^iidry  for  the  VlCS  <>item  consists  of  la$H 
frodu'c  cOmmandc  ;^nd  wc^fls  or  ohfa«es  for  descfjbtng  the 
m.ip  fe;itorc.  These  ohrases  a e (ar.iuiar  content  phrases 
used  by  map  makers  and  are  conta;r4d  m a document 
produced  jotnlly  by  the  Department  of  Commerce  and  the 
Ocoartmont  of  Defense  [US.  Oept.  of  Commerce,  1975} 
Sonif?  evamplfts  'rom.  Ihic  document  are  shO'^n  m figure  5. 
We  have  chosen,  «n  cooperation  with  RACC,  691  phrases 
from  this  documenr.  A 77  pnrase  scosel,  ust^  lo  the 
descr-ption  ot  features  m the  c<ass  d»a»nj^a.e.  has  oeen 
chosen  for  test  purpetes.  The  first  fe^  iir»es  of  the  tasR 
dictionary  are  shown  m figure  6 

The  choice  of  grammar  15  dictated  both  by  the  nature 
of  the  task,  eg.  the  cJescriptiOn  of  map  features,  ano  by  the 
desired  user  interactions,  eg.  user  commarvds.  A factor 
relating  to  user  5alisfaction  is  grammatical  constraint  A 
g-'a^'iniar  with  hign  constraint  impi-cs.  in  general,  fewer 
recognition  errors  and  therefore  greater  satisfaction  Care 
must  be  taken,  howc'-cr,  to  not  cof\sirain  the  grammar  so 
""uch  that  interaction  cecomes  unnatpra*  for  the  user. 

There  arc  several  ways  of  moosing  grammatical 
structure  on  l^*e  phrases  wricn  make  up  the  verbal 
description.  We  are  cur-entiy  eipen-neniing  with  two 
methods,  wnich  rcprcsei'l  tr>o  eilremes  of  constraint.  The 
*irc.t  method  'S  unstructured  where  any  pnrase  may  be 
followed  Ov  ony  phrsse.  1 e.  not  consirant  This  g'ves  the 
user  complete  f'eedom  lo  desc'’;oe  the  map  feature  »n  the 
most  natural  way  S>nce  there  are  other  methods  wh.:ch 
a-Ow  the  naturalness  but  a. so  have  some  constraint,  this 
r-oc1<?  is  used  fO'  fhn  nvestigotiOn  ot  what  accur.ices  are 
dtlamabl?  m the  worst  case.  If  accuracy  is  adeouale  m this 
case,  then  it  wiit  be  more  than  aoequa'e  in  situations  with 
groiitcr  constraint.  The  second  rpe'hod  »s  complete 
constraint,  or  iree'l'ke,  where  each  description  is 
represented  by  a path  from  :hc  'OOt  of  a tree  to  the  one  of 
its  tcavet.  In  this  r"ethOd  rrer^es  reprjsent.ng  all  DOSS-blC 
ci'.Oices  at  a nooe  of  the  tree  are  shown  to  the  user.  After 
one  of  these  poss  bie  utterances  s socmen  and  recognijed. 
the  system  uses  the  recognized  phrase  to  move  tc  the 
appropriate  new  node  a^d  presenis  the  necl  mer-u 
according  to  the  choices  at  the  r>ew  rede.  The  f.rst  menu 
(loo  c'  root  riocej  presentee  10  ;re  user  -s  snown  m tigure 
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figure  6. 


Evample  showing  d'Ctioniry  formit  for  the 
carlograpny  task 
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Figure  7.  First  inenu  presented  to  the  u$ir  after 
tracing  map  feature- 


7 This  menu  describes  the  major  class'ficitiofi  Of  the 
feature  being  described  Each  menu  contains  "restart*  and 
"backup*  as  possible  verbal  commartds.  Restart  means  go 
back  to  the  root  node  ot  the  grammar  tree  and  start  the 
cu"^cnt  description  again  Backup  means  move  back  to  the 
previous  r%cde  of  the  tree  Tnrs  commaisd  be  used  when  a 
error  was  ercounteroo  As  the  description  ts  entered 
verbally,  the  recognized  phrases  are  placed  on  1r>e  display, 
rsear  the  graphical  desc'  pi'on,  for  verification  The  final 
menu  contains  ”ok",  "accept*,  'backup*,  and  “restart*  as 

pVfuS.wC  •■'“ipU.S 
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r«ither  o(  thc^e  mctnods  ^or  gr^mmaticai  structure  is 
viewea  bemg  entitei/  ippfopria.'e  to  tasK.  Anotnor 
mplhoo  vuhicn  inteno  to  mvestigaie  is  an  unoroereo  tree* 
like  scheme  whc'e  eaci'-  desc^’iption  is  a path  thru  a tree 
structure,  but  phrases  can  be  e'^tered  a^y  order  aop  the 
user  need  supply  only  enough  ot  the  path  to  ma<e  -t  unique 
A variation  allows  features  to  have  certain  detaott 
attributes,  eg  "river"  -mpiies  "natural"  The  oefauit  would 
be  used  to  construct  the  unique  description  unless  some 
other  counteracting  choice.  Such  as  "man-rnaoe*  were 
rrentioned. 

The  Vies  syster.»  was  first  oemonsirated  'n  September 
19  76  after  less  than  a mar.*n>onfh  ot  effort.  Pecent 
emphasis  has  been  on  investigaticrx  ot  various  language 
studies.  \A^hiie  no  extensive  accuracy  studes  have  been 
made,  it  appears  tha!  98^  accuracies  are  attainable  with 
moderate  grammatical  constraint. 

DISCLGSICN 

The  research  reoerted  represents  initial  progress 
toward  the  dove>oprr.en!  ot  a system  comoming  visuat  and 
verbal  data  acquisition  for  the  cartography  tasK.  We  have 
shown  lhal  a new  task  can  oe  constructed  m a relatively 
short  hme.  The  sysfem  is  sfiH  m *ts  infancy  and  many 
irvteresling  researci'i  prcDiems  remain  irt  vocabulary  jnai/sis 
a'Ki  design.  language  a 'a»/s's  and  ces'gn  [Gooomao.  i976> 
effects  of  language  structure  and  user  discourse,  interactive 
techniQ-ies.  arxJ  the  irwostigation  of  recognition 

ChardCfer>Stics  unoer  var*OuS  vocabulary  grammatical 

complexities  We  iock  forward  to  pursuing  these  areas  of 
research. 
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A8STRACT 

The  Harpy  speech  recognition  system  wQfr.«  optimaliy  when 
it  "knows"  the  speaker,  t.e.  when  it  has  learned  the  s^eaner 
dependent  characteristics  (speaker  dependent  paran\aters) 
ot  the  speaker.  There  a^e  three  methods  o(  (earring  these 
parameters.  Or^  way  is  to  ger>erate  them  from  a tet  of 
tratrMng  data  which  covers  all  the  aHophones  that  occur  m 
the  t-ask  language.  A second  method  is  to  use  “speaker 
indepertder\t“  parameters  with  a resulting  reduction  in 
accuracy  performarKe.  Since  it  is  irvcdnveniant  for  a ‘new" 
speaker  to  say  a set  of  trair^mg  data  before  us'ng  the 
System  arui  the  low  accuracy  with  speaker  iryiependent 
parari'cters  is  unacceptable,  a thiro  method  has  been 
devised  to  alicw  the  systern  to  dynamically  learo  the 
speaker  dep>cr%<3ent  parameters  while  uS'ng  the  System.  The 
new  speaker  starts  with  a set  of  speaker  independent 
parameters  These  parameters  are  then  altered  after 
correct  recognition  (which  can  be  fo'ced  it  necessary)  to 
match  the  spoken  utterance. 


lNTK)OUC«fON 

^his  piper  presents  a method  by  which  the  Ha'-py  is  able  to 
adapt  to  non-famdiar  speakers  The  first  section  gives  a 
short  description  of  the  Harpy  s/stem.  Ms  data  structures, 
ano  its  cur-’ent  perforriarxe.  The  foiowing  sections  discuss 
the  speaker  variability  issue  and  several  approaches  that 
have  been  taken  tow?»'ds  Ms  soiul  on.  These  approaches 
include  speaker  specific  tuning.  spea*ie'  independent  lunmg, 
and  dynamic  speaker  adaplatioa  The  lost  section  discusses 
hew  these  averaging  techniques  can  aiso  De  used  >n  isolated 
word  recognition  systems. 

Th£  HARPv  SYS'cM 

The  Ha-'py  system  is  the  first  s/stc-'  to  be  demonsl'-aled 
With  a vocabulary  of  over  ICCO  wees  The  system  wes 
Oerronstr^led  at  the  comoid'Cn  of  fhe  f;ve  year  Advanced 
Research  Projects  Agency  (AROA)  speech  research  project 
in  Septernber,  1976  It  had  a sentence  accuracy,  across  fi/e 
speake''s  icoth  r-^ie  a.nd  ‘ema'e;.  df  91“^  ano  ran  -n  about  30 
M’PSS  (a  s rr  :iions  of  machine  instructions  executed 

per  second  oi  speech)  Since  that  time,  imorovements  nave 
been  mao0  n the  speed  ot  the  syster-  The  current  system 
r.rn^  n less  than  7 WlrSS  system  15  a fecogn.tion 

System  rather  tnan  an  u"3ef stanemg  system  s nc»  M jses  no 


semantic  knowledge  about  the  task  in  decc>du>g  the 
utterance,  hlowever,  there  are  several  other  sPieces  of 
knowledge  m the  system  such  ts  syntactic,  lexical,  word 
juncture  phenomera,  soeakcr  charactanstics.  and  intrinsic 
phoneme  durations  (sec  LOwerre,  1976  for  complete  details) 

In  the  Harpy  system,  the  syntactic,  lextcat,  end  word 
juncture  kr>owiedge  are  combined  together  into  one  integral 
network  representation  similar  to  that  of  the  Origois  system 
(Baker,  1976)  The  syntactic  knowledge  'S  specified  by  a 
context  free  set  ot  production  ruiea  tor  the  task  language 
A dictionary  is  used  to  represent  the  laxical  knowledge. 
The  dictionary  contains  symbolic  phone  speltif^s  ar^d 
specifies  alternate  pronunciations  of  the  words  in  t)>e  task 
>angu£gc  Word  juncture  rules  are  also  included  ir  the 
network  to  accourit  for  inter-word  phonetic  phencmena. 
The  .letwork  consists  ot  a set  of  states  and  .nler-state 
pointers  Each  state  has  associated  with  it  phonetic,  lekicat, 
and  duration  ntormation.  The  pointers  indicate  wh*t  states 
may  follow  any  g*ven  slate.  Two  special  states  in  the 
network,  Ine  nitia‘  state  and  the  tma*  state  indicate  the 
starting  point  and  ending  point  for  ail  utlerarjes 
respectively  The  network  is,  therefore,  a complete  (and 
pre-compiicd)  representaiion  cf  ai)  possible  pronunciations 
of  a!l  possible  vl^crances  m the  task  language  This 
network  15  used  to  gu’de  the  recognition  process. 

The  recognition  process  ot  the  Harpy  system  is  based  on 
the  Locus  model  of  search.  The  Locus  model  rejects  ail  out 
a narrow  boarr*  ol  paths  around  the  most  likjly  path  througn 
the  network.  These  "best*  paths  are  searched  m parallel 
with  one  pass  through  the  speech  data  and  there’ore  ooes 
net  require  backf'acking 

T.ne  following  is  a snort  description  cf  the  recognition 
process:  The  utterance  'S  d'g'tizad  at  10  KHz.  Tnis 
continuous  Signal  'S  seg'^er.ted  into  consecutive  acouslicai'y 
S!r»^i!ar  sourvfl  on-.ts  (based  on  distance  meas.ires  of  Ifvc  data) 

and  autocorrelation  values  and  linear  predetor  codmg  >Lr^) 

coefficients  are  extracted  for  each  segment  The  segments 
i'e  then  i^.apped  to  the  network  slates  based  on  the 
prpbabMity  of  match  (O-stance  rratch)  ol  the  lPC  data  and 
the  expected  phones  ot  each  slafe.  Tr.e  malching  ot  the 
‘^PC's  and  the  network  states  is  accompl.sheo  by  ute  of 
pho.ne  templates.  The  temp'-ales  contain  the  idealized 
parameters  for  each  pnone  f'at  occurs  m Irve  network 
states  and  they  rray  be  eMher  speaker  specific  or  speaker 
indepcndert.  Th«  mcl.'ic  used  iO'  this  matching  is  Itakura's 
mininium  prediction  res  dua*  er^^or  (see  Itakura,  19/5) 
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Tk«  m«poir>g  schema  used  is  4 gr^pK  search  >r. 

which  heuristics  arc  c^d  to  raduca  the  number  of  paths 
that  are  checked  The  resuit  <s  that  onty  a *best*  paths 
are  searched  m parjtiei  during  the  recognition  processes 
thus  greatly  redocmi  the  computational  overhead. 

Pecults  The  currant  system  achieves  a sentence  accuracy  of 
90  02  and  a word  accurKy  of  94  32  on  a 1011  word  leek 
ar>d  runs  ir.  6.8  MIPSS 


Sf^AKER  adaptation  IN  T«  HARPY  SY$TEU 

Speaker  variability  Speaker  variability  ger>erally  occurs  m 
three  forms,  dialectic,  contextual,  artd  acoustic.  Dialectic 
variability  involves  changes  in  the  pronunctation  of  words 
among  speakers.  Contaxtuai  variability  mwotvas  changes  m 
word  pror^unclat  on  do  to  the  context  of  the  wards 
Acoustic  variability  results  from  vocal  tract  changes  among 
speakers.  Eithoi  or  all  types  Cf  <.ariabi‘ity  can  occur  when 
Changing  speakers  The  Harpy  system  attempts  to  recogrsize 
these  different  variabilities  and  to  separate  the  effects 
made  by  each.  Dialectic  var  abihly  is  an  effect  across  a 
broad  group  Of  speakers  and  the  vaf>aDii.t,>  <s  encoded  into 
the  ie)(icon.  k4any  dialects  can  be  encode..  'Mo  the  leekon 
Or  different  lexicons  can  be  used  for  aifferent  dialects.  The 
current  Harpy  system  uses  the  “mid-wesfern  American* 
dialect  of  E/>g!ish.  The  contextual  vanabilify  n nandled  <n 
the  word  iuncture  phenomena  rjtes  and.  to  a lesser  extent, 
m the  lexicon  itself.  The  acoustic  variability  iS  a speaker 
dependent  phenomenon  and  can  be  separsted  from  the 
Other  types  Of  variability. 

Approach  to  sr>eaker  var;abil>(y  Kf'.any  proposals  and 
attempts  have  been  made,  from  such  groups  as  SOC,  6BN, 
Lincoln  Labs.  etc.,  as  to  how  to  handle  the  speaker 
variability  problem  These  proposals  include  such  ideas  as 
vowpl  formant  normahzations  as  an  attempt  to  determine 
speaker  independent  characteristics  of  the  speech  signal 
The  Harpy  system  handies  speaker  variability  by  the  use  of 
phone  templates  to  capture  tne  vocal  tract  cKeracteri$t>ct. 
Me  achieve  this  by  identifying  all  the  unique  sounds  that 
occur  in  the  task  language  (called  phones)  U is  important  to 
realize  that  these  phor^es  may  or  may  not  bear  a 
resempiarce  to  wnat  rnay  be  usually  th^ought  of  as  a 
pnonelic  souns  m the  English  >argu«ge  For  example,  there 
are  usually  s?vera!  occurrences  of  orve  vowel  (ailophones)  «n 
Our  set  of  pnone*  each  ot  which  has  a unique  name.  Also, 
there  could  be  a single  phore  which  represerts  what  is 
usually  thought  of  as  a combmaticn  of  phones  (e.g  the 
phone  “WH*  represents  the  characienslus  of  the  aspiration 
sound  when  pair  "K  W”  that  occures  together  as  m the  word 
“queen")  Each  of  (he  phones  used  m the  Hai-py  system 
represents  one  unique  ohon-hc  sc-nd 

Phonetic  knowledge  i^  the  Ha'py  s» atem  The  Harpy  system 
uses  a ohoneiic  dictionary  (a'ong  with  word  juncture  rules) 
to  'cprcsent  the  lexicon  of  the  task  language.  The  spellings 
>rt  the  dictionary  are  strings  of  phones  (along  with  a special 
Syntax)  which  are  used  to  represent  primary  and  alternate 
pronunciations  of  the  words  m the  lexicon.  The  phonetic 
dictionary  is  a representation  of  the  actual  realizations  of 
the  task  language  words  rather  than  a pronunciation 
dictionary.  A set  of  speaker  dependent  pnene  templates 
(one  per  phone]  IS  used  lo  maUn  tne  sy'nooUc  ie*-con  to  Ire 
actual  acoustic  Signal  The  phones  of  tne  lex’con  represent 


ift  the  unique  pnorwht  stnmds  fhet  eecur  m the  lask 
language.  Since  the  lexicon  contains  symbolK  ipetlings 
which  are  speaker  uvjependnnt  end  there  is  a one  to  or>e 
mapping  of  the  templates  to  the  phones,  the  ecoustic 
■peaker  vanabili’y  can  be  har^Jisd  effectively  by  us«ng  a 
ursique  set  of  templates  for  each  speaker.  T)\e  templates 
model  speaker  dependent  voce)  characteristics.  For 
example,  ',he  dictionary  spelling  for  “CONCERN*  is  “<•“  (-.O),-) 
(K,0)  (IH7,ITO)  N S ER  (N,DX)"  Optionel  paths  are  erKlosed 
Within  parenthesis  and  are  sepvated  by  commes  (the 
represents  the  null  option).  The  spelling  is  interpreted  as 
either  a voice  bar  (*»>“)  followed  by  an  optional  silence  (*-*) 
or  just  a Silence,  followed  by  an  optional  *k*,  followed  by 
either  a "IH?"  or  *1H3*,  followed  by  an  “N*.  followed  by  a 
**ST  followed  by  an  "ER",  followed  by  either  ar>  or  “OX*. 
See  McKeown,  1977,  for  an  exampto  networiu 

Averaging  ^ template  exemplars  The  success  of  the 
speaker  dependent  phonetic  templates  depends  of  the 
ability  tc  average  many  exemplars  of  each  phor^e  together 
to  generate  each  template.  This  averaging  enables  the 
automatic  cancelation  of  errors  (provided  they  a/a  smsil). 
Since  the  template  is  an  average,  there  is  rw  r^eed  to  find 
the  single  "oeat*  exemplar  that  best  fits  ail  occurrahces  of 
the  phone.  The  averaged  template  w>il  usually  match  all 
exemplars  of  the  phone  m the  training  data  to  i high  degree 
Of  accuracy.  If  a match  of  an  exemplar  m the  training  data 
is  too  far  f.'pm  the  average  template,  then  this  indicetes  e 
miss.ng  phone. 

The  metric  used  by  the  Harpy  system  is  Itakura's  minimum 
prediction  residual  error  of  the  LPC  data  A method  was 
needed  to  average  samples  logether  that  could  be  used  for 
gerverat.ng  the  templates  for  this  metric.  The  method  we 
use  IS  to  sum  the  autoco'rctatipn  data  of  the  samples  that 
are  used  m generating  the  template.  The  justificalion  of  this 
•s  lhal  the  LPC's  are  ;ndepcndenl  el  the  number  of 
autocorrelation  samples  that  are  used  to  generate  them.. 
The  obvious  danger  is  that  non-similar  sounds  may  be 
averaged  resulting  is  a poor  spectrum  This  is  a real 
problem  and  is  handled  by  a semi-automatic  procedure  for 
generation  of  the  pnones,  templates,  lexicon,  and  word 
juncture  rules  described  beiow. 

Speaker  specific  tunmg  Tne  phones,  templates,  lexicon,  n c 
word  juncture  rules  are  generated  from  a set  of  framing 
data  that  contains  occur'cnces  (and  hopefully  all  contexts) 
of  all  the  words  m the  'ex. con.  A semi-automatic  iterative 
procedure  '$  used  to  generate  (or  more  precisely,  update) 
these  knowledge  sources.  There  i$  a “chicken-egg"  proplem 
With  this  iterative  procedure  m that  the  data  sources  must 
already  exist  m Order  to  update  them.  The  gerxeration  of 
the  initial  knqwiecige  sources  i$  a iediOu»  manual  boot- 
strapping procedure.  The  training  data  must  be  carefully 
hand  labeled  (both  at  the  wo.'d  level  and  the  pnone  level) 
and  initial  guesses  are  made  about  what  phones,  word 
spellings,  juncture  rules,  etc  are  reeded.  This  manual  effort 
IS  the  main  bottle-neck  for  developing  larger  vocabutary 
system:  Automatic  methods  must  be  developed  be'or# 

larger  systems  can  be  atterrpted 

Vhc  foltowing  IS  the  serr  -aulomalic  p'ocedu.'e  used  to 
update  the  data  sources;  The  Harpy  system  is  run  m a 
forced  recognition  mode  with  a previously  generated  set  of 
templates  (which  can  be  from  some  other  speaker)  .0 
produce  a parsing  of  the  pror.es  to  the  acousl.c  data.  Th.s 
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force<j  recognition  Can  oa  Oona  titKar  by  using  a umqua 
natwOfK  for  each  utterance  fwhi;h  rep'esenfs  only  the  one 
utterance)  or  by  corvsidering  onfy  paths  m a large  nebtrork 
that  represent  each  sir^ie  utterance.  The  parsings 
generated  from  the  forced  racogrtition  runs  are  used  to 
locale  the  autocorrelation  data  tor  the  averagir>g  of  the 
templates.  After  the  averaging  ii  completecl,  a tsew  sat  Of 
templates  is  generated  and  used  to  again  nan  the  trairung 
cycle.  This  cycle  is  run  several  limeo  untit  the  -empiates 
converge  If  the  templates  do  rtct  converge,  then  thu 
iryjfcates  an  error  m either  tf^  teiKOn  or  word  (uncture 
rules  or  a missirxg  phone  which  rrujat  be  manually  analyzed 
and  corrected 

Speaker  ir.de  pendent  tuning  The  speaker  dependent 
templates  are  an  averaging  of  r^any  phone  exemplars  for 
each  tempuie.  Since  there  «s  a unique  set  of  templates  for 
each  speaker,  they  capture  the  irxiividuaf  vocal  tract 
characteristics.  This  idea  of  capturing  vocal  tract 
characteristics  by  the  use  of  templates  can  be  extervded  to 
muUiple  speakers.  Wl>en  a number  of  these  speaker 
aepertdent  sets  of  templates  ere  generated,  ar\other  set  of 
templates  cars  be  generatec  from  all  of  them  by  a s:m<lar 
averaging  technique.  This  set  of  templates,  since  they  are 
an  averagirtg  of  several  speakers,  will  be  speaker 
irvdependent.  The  performance  with  speaker  independen* 
templates  wilt  of  course  oe  •ower  than  with  the  speaker 
dependent  templates  For  example,  one  experiment  done 
wu'h  cor^nected  digits  give  the  following  result:  Ten 
speakers  (including  males  and  femaiest  were  used  to 
produce  ten  speaker  oapervden!  se!s  of  templates  The 
average  word  accuracy  for  ail  ten  speakers  (when  tested  on 
the  speaner  dependent  templates  \/i!h  a total  of  1000  three 
word  utterarures)  was  93Z.  Tr\esa  ten  template  sets  were 
then  used  to  generate  a set  of  speaker  irsdependent 
templates.  These  same  te«  speakers  plus  ten  new  speakers 
were  then  tested  with  the  system.  The  word  accuracy  for 
all  30  speakers  (on  1300  utler.:^ces>  w»s  9311  An 
interestirvg  observation  <s  that  there  was  no  significant 
difference  between  th.«  accuracies  of  the  ten  speakers 
whose  templates  were  used  to  generate  the  speaker 
independent  set  arid  the  ten  new  speaxers. 

Cs  namic  speaker  adaptation  The  high  error  rate  (Tl)  with 
the  speaker  irvdepenqenr  templates  maxes  this  aiternatue  to 
the  handfing  of  acoustic  var;aoi' ty  unacceptable.  Further, 
the  training  cyc-e  men,  oned  earlier  to  generate  the  speaker 
dependent  templates  .s  inconvenient  do  to  the  large  amount 
of  training  data  r\eedec  and  i$  co.“pulatiOna!ly  expensive  A 
third  scheme  was  devised  which  allows  a new  user  the 
immediate  use  of  the  system  bat  also  allows  for  the  speaker 
oepenoenf  vocal  charjcle-istics  I’his  is  the  dynamic  tumr^g 
ct  the  speaker  templates  A new  speaker  *o  the  systeni 
starts  with  the  set  of  speaker  independent  templates.  The 
system  w»il,  upon  all  correct  f ecognitions.  automal'Calty 
.verage  thiO  autocorrelation  data  with  the  cor'espondmg 
templates  arvi  update  the  !cn*p'ate  parameters  The  hrsl 
occurrence  ot  a phone  spoken  by  the  speaker  will  replace 
the  speaker  independent  template,  further  occ.^rrc'^ces  of 
the  sane  phone  will  add  to  the  average  Of  *he  temp-ale. 
Th.s  w'!j  .esuit  in  the  phene  template  being  altered  quick* 
for  the  first  Occurrences  of  3 pnone  and  a gradua*  i.'ie 
tuning  of  the  template  by  add't-or’i'  occurrp.nces  oi  me 
phone.  In  this  method,  the  system  qu'Ckiy  adapts  '(ifir  to 
the  speaker’s  acoust  c characteristics  the  system  ^akes 
an  error  in  recogcifion,  one  can  e.ther  speak  the  same 


utterarsce  aga^n  with  the  hope  that  it  wii;  bo  recognized 
correctly  the  secorsd  time  or  the  system  can  be  rerun  on  the 
same  utlererKe  arsd  forced  10  recognae  the  utterance.  To 
force  e recognition,  the  appropriate  Switch  IS  set  and  the 
correct  utterance  iS  typed  to  th#  system.  The  system  will 
then  only  consider  paths  in  its  network  which  represent  the 
Spoken  utterarKe. 

The  error  rate  when  first  starting  is,  of  course,  71  but 
quickly  drops  Off  towards  the  Zl  error  rate  of  the  speaker 
dependent  templates.  The  time  needed  for  the  i^datmg  of 
the  templates  is  aero  during  the  Ktust  resogmtiM  but 
requires  up  to  One  times  real  time  after  recognition 
depending  on  tive  number  of  templates  that  are  updated. 
Therefore,  the  overload  cf  doing  the  dynamic  speaker 
adaptation  is  mimmal. 


DISCUSSION 

Summary  !n  this  paper  we  have  considered  several  sources 
of  variability  <ri  the  connectect  speech  signal,  i.a.  di^ectic, 
contextual,  ar^  speaker  dependent  variabHity,  arxf  Ascribed 
how  the  Harpy  system  attempts  to  cope  with  ell  these 
sources  of  variability  The  dialectic  and  contextual 
variability  are  encoded  into  the  lexicon  ar^  word  juncture 
rules.  The  speaker  dependent  sources  of  vanabihty  are 
handled  by  averaging  phone  parameters  (i.a.,  the 
autocorrelation  coefficients,  f>ol  the  L?C*s)  from  among 
several  exemplars  cf  a given  phone  by  the  same  speaker 
(for  speaker  specific  templates)  or  from  many  speakers  (for 
speaker  iricependent  terr.plates).  Iri  the  case  of  dynamic 
adaptation,  a set  of  soeaxer  independent  templates  are  used 
initially  and  the  system  auromat'caliy  alters  the  templates 
during  use  to  adapt  to  the  specific  speaker. 

It  appears  straight  forward  to  adept  the  above  techr\rOuas 
for  isolated  word  recogn.tipn  systems  also.  Given  several 
tra-ning  samples  Of  the  same  word,  one  cm  align  the  speech 
Signal  by  dynamic  programming  techniques  and  average  Ihe 
autocorrdaton  coefficients  as  m the  connected  speech  case 
Since  this  averaging  wouio  be  independent  of  word 
reprosentation  used,  i.e.  whether  one  uses  segmentation  and 
phone  templates  to  represent  words  or  ihe  conventio.nai 
brute  ^orce  word  templates,  one  can  st:*i  use  the  above 
ave* aging  technque  to  generate  better  templates 
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abstract 

We  have  been  attempting  to  produce  furthef 
bapvdwidth  rodiKtfOn  in  LPC  baled  analysis-syotneits 
technlQue*  by  uimg  the  segmentatr^n  and  labeling 
• u^ed  in  the  Kafpy  anp  Heafiayli  tystem^. 

Preliminary  rc&ults  indicate  that  a factor  of  3 to  5 tuflher 
reduction  in  bandwidth  might  be  possible  using  segmentation 
and  labeling  in  conjunction  with  LPC  vocoders. 


INTROOUCTHDW 

An  importanl  application  of  speech  analysis ^syntbests 
IS  digital  voice  transr^issioru  Reai'hme  trarumtssion  «l  low 
bandwidths  can  oniy  be  achieved  through  efficient  analysis 
and  encoding  techniques.  While  present  analysis  n'cthods. 
based  on  signal  processing  techniques,  nave  been  used 
successfully  lo  obtain  barvswidth  redactions  of  ove'  an 
order  of  magnituoe.  further  msprovemcnt  ‘S  possible  if 
higher  level  properties  of  speech  arc  aiso  taserx  mto 
account.  In  tns  paper,  we  dcfrtonsi'aJe  how  segmentation 
and  labeling,  two  techniques  commonly  used  in  connected 
speech  recogrtilion,  can  be  applied  to  vocoder  systems  as  a 

moans  of  improving  codir»^  efficiency. 

In  the  rema  r ng  sections,  we  dcsc^  bo  segment*.!. on 
ar>d  labeling  techrvqoes,  and  tneir  use  ‘n  vocoder  systems. 
Results  frpm  three  different  vocoder  s>mulatons  based  on 
these  techniques  are  p'esenifd  arvJ  eva’va’ed  V^e  ihen 
consider  sone  of  !ho  practical  aspects  of  real-time  speech 
transmission  jsmg  these  methoos.  Finally,  the  advantages 
and  disadvantages  Of  high  level  speech  processing  as 
applrod  to  vocoaers  are  discussed 

APPROACH 

Our  goal  in  th  s study  was  to  evaluate  the  usefu-ness 
of  eegmentalion  aro  Japolmg  as  techniques  lor  improving 
vocoder  codmg  efficiency.  To  accorrphsh  th's,  two  vocoder 
Simulations  using  each  of  these  techniques  separaleiy,  and  a 
vocoder  simuSatior  which  cor-.bired  the  techn*qu*5,  were 
run.  The  results  were  compared  w»th  inose  ob!a»rsed  using 
converstional  pararrioter  encoding  methods,  and  evaluated  *n 
terms  of  tandwiCIh  reduction  and  quauly  ot  syrthelic 
speech. 

Of  the  several  techniques  for  speech  analysis  that 
exist,  this  paper  considers  cniy  those  based  on  the 
autocorrelation  melhcd  Of  unear  predfCtion.  A compieie 
vocoder  sin'.uiation  eased  on  this  tecnn>aue  has  already  been 
developed  by  Ma'Kel  and  Griy(Mdrxci  and  G.*iy.  *570]. 
SifKC  a detailed  a scussion  can  be  found  m this  reference, 
w*  consider  only  those  aspects  relevant  lo  the  banowicth 
problem  here. 


Analysis  oarameters  m autocorrelation  based  linear 
ored  cfion  systems  consist  of  prtch  period,  a 
voiced/unvoiced  decision,  amplitude  information,  and  parcor 
coefficients.  These  parameters  are  generally  eiKOded  mto  a 
mtmrnai  bit  represer.tafion  and  trananitted  at  a constant 
frame  rate.  The  system  on  which  our  c<^nparisons  are 
based  uses  a frame  rate  of  100  frames/sec,  where  each 
frame  consists  of  200  speech  samples.  A totel  of  64  bits 
ere  allocated  to  the  14  pa/cor  pa/ameters,  which  are 
quantized  as  described  m [Uerkel  and  Gray,  1974],  Pitch 
period  ar%d  the  voiced/unvoiced  decision  ere  encoded 
together  in  6 bits,  end  the  amplitude  if  coded  into  5 bits. 

S£:G1^MT-C0C3CR 

Classically.  m*ormatiort  concerning  the  vocal  tract 
shape  IS  transmitted  >n  the  form  of  parcor  parameters  once 
per  analysis  frame.  Sp>eech,  however,  ca'Y  be  segmented 
into  events,  -or  the  duration  of  which,  vocaJ  tract  shape  may 
be  considered  approximately  constant.  Cases  where  this  is 
not  true,  such  as  glides  and  diphthongs,  may  be 
approximated  by  a series  of  iho/tcr  segments.  Therefore,  it 
should  be  possible,  wtheut  significant  degradation  tn 
synthetic  speech  quality,  to  transmit  parcor  parameters  once 
per  segment,  rather  than  once  per  frame.  S>r*ce  segment 
duration  is  relatively  long  compared  with  a.naiysii  frame 
length,  a savings  >n  the  number  of  bits  needed  to  encode  toe 
analysis  parameters  should  result.  A vocoder  Simulation 
based  on  this  hypoinesis  was  developed 

Segmentation  is  orefermed  usir\g  algorithms  oeveipped 
for  the  Hca'say  speech  recognition  sy$ltm[GoiOb«rg  ano 
Reddy,  1976]  Three  stages  are  involved  in  the  Overall 
process:  paraiYetrijalicn,  segmentation,  arkd  ciassiticatron. 
The  first  step  m parametriiation  i$  to  generate  smoothed 
and  differenced  waveforms  irom  the  sampled  speecn.  Next, 
peal*  to  peak  amplitudes  and  zero  crossing  counts  a/e 
extracted  from  each  waveform  once  per  cant.secOrsd  Ot 
speech.  Segmentation  is  based  on  ir\ese  parameters. 

Segrnent  boundaries  are  determined  by  successive 
subdivision  of  the  waveform.  Firal,  ti'ences  and  unvpiced 
fricatives  are  deteclec*  by  a throshOiOmg  techn.que.  ^iex^, 
the  remaining  segments  are  divided  \<here  ^.gnificant  Oios  m 
the  smoothed  peak  to  peak  parameter  occur  A reg  on 
growing  techniQ.,e  <t  then  applied  to  further  sobd  vice  the 
segments.  Fmaiiy,  the  resulting  segments  may  cpilOnaiiy  be 
classified  m terms  of  manner  of  arlKuiaton.  CeciS'On  ^uiei 
based  on  the  averaged  parameter  values  for  each  segment 
are  used  for  th.s  purpose. 

Operation  of  the  vocoder  'S  roiative  y slra  ghHorward. 
Speech  is  segmented  as  it  er.teri  the  system.  When  a 
segment  boundary  0<:urs.  parcOr  parameters  for  that 
segment  are  caicjialed  oy  def:nit»OA,  »'■'  wilhn  a 

ttgmer^t  sheud  nave  s-mnar  spectrii  prooeriias, 


Reprinced  from  ;977  IEEE  Coni.  ASSP,  28-32. 
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riguf  A 1.  Spectral  mismatch  resulting  frcm  interpolation  ot  parcOf  coetficientt  tt  segment  bounOeries. 
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th«s  is  oct  elways  tha  ca^e.  Near  tegmem  boundaties.  the 
vocal  tract  >9  cKar^rn^,  and  cannot  be  aciumed  to  have 
constant  resonances.  To  eUminatt  possible  errors  due  to 
these  changes,  the  parcor  cc^fficients  are  computed  at  the 
segment  m,dpOinl.  Once  caicuiateo,  these  coe'f^ients.  a>ang 
erith  the  segment  OuratiOA,  are  transrvtted.  Fitch  and 
amplitude  are  then  extracted  from  each  frame  in  the 
segment,  encoded,  and  transmitted.  Thus,  with  this  scheme, 
pitch  end  amplitude  are  still  transmitted  a!  tne  constant  rate 
of  once  per  frame,  but  parcor  coefficients  are  transmitted  at 
the  rate  of  once  per  segment,  which  Is  not  necessarily 
cor>s(ant. 

Except  at  Its  boundaries,  the  same  set  of  parcor 
coefficients  is  used  to  synthesiae  speech  for  each  frame 
within  • segment.  Near  boundaries  variation  in  the  parcor 
coefficier^ts  due  to  vocal  tract  changes  rnust  be  taken  into 
eccount.  Good  results  nave  been  oblaned  using  simple 
lirsear  mterir'ilatica  For  most  segrrenis  *1  is  adequate  to 
Interpolate  over  5 centisacsnds.  from  2 centiseconds  before 
the  segment  boundary,  to  2 centisetoncs  after.  For  shorter 
segments,  indicating  rap'd  changes  m the  spectral  structure, 
interpolation  »s  don*  fr*-  **><  sefr^O''!  midpnifj. 

The  effects  of  parcor  coeffrcer.t  interpolation  are 
illustrated  in  Figure  1.  Th«s  figure  shows  the  spectral 
ersvolopes  for  a transition  from  or>e  segment  to  the  next. 
The  darker  curve  represents  the  conventional  Synthetic 
speech,  the  lighter  represents  the  speech  synthesized  from 
interpolated  parcor  cocf ficcents.  Note  fhat  although  the 
peak  amplitude  and  shape  differ  slightly,  the  peak  locations 
are  rseaMy  identical. 

Figure  3 shows  a digital  spectrogram  for  the 
utterance  "The  area  Tm  interested  m is  understanding." 
synthesized  with  the  Seg'^ent -coder.  For  comparison 
pc^poses,  a digital  spectrogram  of  (he  utter ar^e  synthesized 
with  conventional  meihods  'S  srown  m Figure  2-  As  can  be 
seers,  the  spectrograms  resemble  each  other  closely  In 
informel  listening  tests,  the  synthetic  speech  generated  w<th 
percor  parameters  transmitted  only  cnee  per  segment  was 
nearly  trsd'Stinguishab^e  from  that  gerserated  with  pa'cor 
parameters  transmitted  every  frame. 

The  degree  of  improverr.eni  m C0<j>ng  efficiency  Will 
vary  from  system  to  system,  dcDer>d*ng  on  frame  rale,  and 
the  precision  to  which  each  of  the  parameters  ar#  encoded- 
For  the  system  ciesenbed  eari'cr,  a total  of 
(6e5*€A>i  100-75C0  bits/sec  are  required  to  encode  'he 
analysis  parameters.  L)s>ng  segmentation,  p<tch  period  and 
amplitude  information  are  still  transmitted  for  eKh  frame. 


but  parcor  coefficients  ara  transmitted  only  once  per 
segment  Another  parameter,  the  segment  duration,  enist 
also  be  transmitted  with  each  segmerst.  Allocation  of  A bits 
for  this  parameter  allows  for  segmer^t  lengths  up  to  16 
centiseconds.  Segments  exceeding  thus  length  art  rarely 
encouritered,  and  can  easily  be  split  mto  multiple  segments. 
On  average,  the  segmentation  algorithm  produces  15 
segments  per  second  of  speech.  Thus,  tfw  total  pit  rate 
needed  for  this  scheme  is  (6^5hlOO*{A#64hil5»2120 
bits/sec.  This  represents  improvement  by  a factor  of  about 
3.5  over  the  conventior^al  method. 

Reductior\s  of  tNs  ordee  have  been  obtained  tn 
conventional  vocoders  by  usu^  reouced  frame  rate.  Pathar 
tharx  transmitting  one  frame  per  centisecpnd.  these  vocoders 
might  transrMt  one  frame  every  3 centiseconds, 
indiscrimiriately  ignoring  data  between  frames.  This  has  a 
smoothing  effect  which  results  m the  loss  of  short  events 
that  may  be  perceptually  sigruftcanl.  Thus,  the  overall 
quality  of  the  synthetic  speach  should  be  lower  than  that 
Ootair^d  with  the  segmentation  schamo. 

LAKL-COOCfi 

A second  techn.que  makes  use  of  an  astumpticn  that 
all  speech,  regardless  of  >ts  consptexity,  can  be  formed  by 
combinations  of  a smeii  number  Of  basic  sounds.  The 
VOPTRAX  speech  synthesizer  is  an  eiamplt  of  one  such 
system  based  on  this  assumptiorv  Associated  xnlh  tich 
sound  IS  unique  formation  of  the  vocal  tract,  and  associated 
with  each  vocal  tract  formation  »s  a set  of  parcov 
coefficients.  If  speech  at  each  analysis  frame  can  be 
identified  and  classified  as  one  ol  these  stxmds,  then  it 
would  only  be  neces:^*/  to  transrmi  a label  identifying  the 
sound,  rather  that  the  entift  set  of  parePr  parameters. 
Since  the  number  of  sounds  is  small,  sigmfkantly  fewer  that 
64  bits  ere  needed  to  encorie  the  label,  and  an  improvement 
m codirrg  efficiency  would  result. 

Prior  to  the  developmeni  of  a vocooer  sunulation.  the 
properties  of  each  sound  must  be  deterrruned  and 
represented  m a format  usable  by  the  system.  A procedure 
to  accompii'h  this  was  developed  for  use  with  the  ^rpy 
sysfem[Lowerrc,  1376J.  Segrrenis  from  several  utterances, 
spoken  by  a particular  speiker,  are  identified  and  grouped 
according  to  their  sound  cUss.  Autocorrelation  coelf'C  ents 
>or  each  segment  are  compjted  and  averaged  ever  a*l 
seements  m the  sa«T>e  class.  For  etch  avenged 
autocorrelation  sequeiHt,  hereafter  referred  to  as  a 
template,  linear  prediction  coefficients,  parcor  coefficients, 
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b^Mffici«Ats[itakLjra,  1975}  ar»  coribtfted.  T>uft 
information  is  mad*  available  to  both  the  transmitter  and 
recetvor  portions  of  t^e  vocoder 

The  task  ol  the  vocoder,  then,  *4  to  determine,  for 
tech  analysis  frame.  wh»ch  template  be«t  matches  the 
speech  si^nat  The  LPC  matching  technique  developed  by 
ttakuraCltakorai  1975]  ^.as  been  u^ed  for  this  purpose.  A 
distance  metne  13  aopticd  between  each  frama  and  aU 
tamptates.  The  best  template,  m term$  of  rntTHmum  dfstarKe, 
is  »eiected-  A laoel  identify'rig  tNs  template,  aiorvg  with 
pitch  and  amplitude  (ntomatron  is  tranim«tted.  At  tt^ 
receiver,  a simple  table  lookup,  usrng  tt^  label  as  an  mdex, 
is  preformed  to  determine  the  parcOr  parameters  Of  each 
frame  From  this  pomt  on,  syr^the^is  proceeds  normaHy. 

Figure  6 3how«  the  spectral  mismatch  between 
originai  spectra  and  the  labels  assigrteo  to  them.  The  darker 
curve  corresponds  to  the  orig-nal  speech,  the  I-ghter  to 
speech  synthes  aed  with  the  iabebr\g  method.  The  curves 
illustrate  typ«cai  spectral  errors  that  occur  with  the  laoefir^ 
method. 

Orsplayed  m Figure  4 is  a d^«tai  spectrogram  of  the 
test  utterance,  synthesized  wifn  the  Laoe‘-co«ief  Thu  may 
be  compared  with  the  spectrogram  of  the  <or^vent*onai 
synthetic  speech  m Figure  2 Although  the  synthetic  speech 
wes  intelligible,  there  was  cQns<oerable  distortton.  We 
believe  that  this  can  be  ei'minated  by  charxges  m the 
template  generation  aryj  matching  algorithms. 

Again,  the  bar^widtn  reduction  afforded  by  this 
techrvioue  depends  on  how  accurately  the  parameters  a*e 
Ouantized,  but  m this  case  it  is  independent  of  frame  rate. 
As  before,  we  base  Our  comparison  on  the  system  des'.ribed 
earlier.  For  this  system  a total  of  6*5»64»75  brts/frame 
are  needed  to  encpoe  the  speech.  For  the  system  with 
labeling,  a labei,  along  with  the  encoded  pitch  ar^  amplitude, 
•a  transmitted  fwi  ca*.’.  V'imc.  To  un-quely  derstily  each  of 
the  96  templates  used  m th\$  s.n  u'atton,  7 bits  were 
•llocatod  for  the  label  Thus,  with  .'abei<fH«  only  6*5*7»1A 
bits  are  needed  to  encode  each  frame  This  represtnU  a 
barsdwtdth  reduction  by  a factor  of  4 

SEGkCNT -label  -CCCC  ^ 

Oearly,  if  only  one  set  of  parcor  <oetttcier«ts  if 
necessary  to  erKOde  the  spectra:  structure  of  each  segment, 
and  if  each  spectral  structure  can  ci«  identified  by  a label, 
then  it  should  be  possible  to  transm,t  omy  ore  lao?:  per 
eegmer^t.  Examination  of  ihe  analysis  parameters  from  the 
labeling  system  reveals  that  this  is  moeed  th#  case.  Most 


frames  within  a segment  wiere  found  to  b«  labeled  with  the 
same  tabet.  Those  tha!  wrere  not,  were  labeled  with  an 
acouslicaify  si.  uiar  label  Cnee  agauv  a vocoder  simulation 
to  test  the  hypothhsis  was  developed- 

The  separate  use  of  segmentation  end  labeling  has 
already  been  d'seussed.  This  system  n merely  1 
comoinalton  of  the  tw-Q  p/evtoos  ones.  After  segmentation, 
the  labeUng  algorithm  is  applied  at  the  midpotnl  of  each 
segment.  The  label  which  beet  characterizes  the  spectral 
properties  of  that  segment,  and  the  segm«M  duration  arc 
eiscoded  for  transrissiOA.  Of  course,  pitch  arvd  amphtude 
tnfarnvaliOn  arc  stiH  transmitted  fOf  every  frame.  Received 
labels  are  first  used  deterrnne  the  parcor  parameters 
associated  with  ea.n  segment,  wmen  m turn  are  used  to 
synthesize  speech  tor  alt  frames  wdNrs  that  segment. 
Ir^terpolation  at  segment  bourvdartes  is  earned  cut  as 
previously  described 

The  spectrogram  for  speech  synthesized  by  this 
system  is  shown  Figure  5.  Note  its  s»milarily  to  the 
spectrogram  fc^  speech  synthesized  by  the  labehng  system. 
This  >%  to  be  expected,  since  it  was  aireedy  determined  that 
segmentation  csl'^cs  sig  nficant  degradation.  Tnp 

d'ffercrtces  between  !h-s  and  the  Other  specl^ograme  are 
due  to  degradation  miroouced  by  label. ng 

Again,  we  calcuiafe  ceemg  efficiency  by  comparison 
with  the  conventtonal  system.  ‘Ahih  this  encoding  sche—e.  a 
total  of  6 bits  for  ptch.  and  5 bits  for  ampUtuoe  are 
transmitted  every  frame  An  adcilional  A bits  for  segr-vent 
duration,  and  7 b<ls  to  identify  the  template  are  |rir.$frutled 
for  each  segmen*.  Us’ng  a Irarr.®  rate  of  lOO  frames/sec, 
and  an  average  of  IS  segments  per  second  of  spee.:h,  a 
data  rale  of  (4*5)xlOC»(4»7;ii5*'i2b5  b»ls/secorvd  ?s 
obtained  This  is  approvimaiety  5.9  times  smaller  tnan  tre 
7500  bits/sec  of  the  conventional  system. 

DISCUSSION 

We  have  shown  that  segmentation  and  labeling  can  pe 
used  *•  a r^eans  o?  recixing  bandwidth  <n  speecn  a'la.yS'^- 
synthesis  systems  5»nce  the  primar/  appi'cal-on  ot  such 
Systems  IS  secure  vO'Co  communtcatiOni.  it  is  appropriate  to 
mention  some  of  the  practical  aspects  of  a vocoder  pastd  on 
these  techniques 

A proplem  arises  wnen  the  vocoder  is  converted  to 
real-trm»  operatiorv  S.nce  analysis  parameters  ic/  eacn 
segiTieni  are  r»ot  trani^  tted  until  the  enti'e  segm-f»t  nas 
been  spoken,  ii  'S  possib'^e  for  the  Syr.tnss'zer  to  co-v^/eie 
synthesis  of  one  seg"'ent  oefore  it  receives  parameters  »or 


’.36 


wtll  Ozcur  To  vO’d  thesa  p*u-.c4  «l  •»  ncca-itary  g 

i^gmcn?  curai'O^  <i*lay  ^'“•  vynrh^^n  hy  tNi 
s-n^i/nt  We  have  aVfjidy  •'"'Ctafea  thaJ  16  c*^!*$«co^-di  <t 
• rea«o«ab*e  t^O‘;ff  to?  «gi»eri  Ow^ati*#*-  !•  th# 

ft/nthesizer  ttg4  frar».ir!.Kpr  by  K*vi  a/i^uAt,  t*us  jr» 

»-^^l»on».‘  2 cifr./iseco'vJs  to  *»iOfw  fof 
conttnuOut  syr.lKei  C ibeeirt  can  be  guattAteed  (ft 
^4t9  not  g &«rtCKjf  b^awbaca  CeUyf  tivc  tTe 

eecoftdjfy  to  nature  to  those  rtorna^'v  tft^Oufrtertd  fft 
basp<J  sys:«mi 

From  fht  <4<scuss'*aft  o'  (abe-  '-'g  •(  should  be  elear  thil 
both  traosmit'ir  aoo  'ecfiv**  wutt  ac<ess  to  tft#  vame  set 
oi  kr  platen.  C-ftce  (he  ttnouies  very  trow  speaker  to 
speaker,  it  ts  -'mprKttcal  tc  'a»e  'hc-m  a iaa  maoeftt  part  crt 
the  sys'em.  Rather,  at  the  Beg»ofi..-^  of  a coftversat*on, 
f«mr»tfttes  for  eac^  s:>«tker  couuj  ta  loaded  *nto  the 
C<Kres|>Oftdmg  l/ansM  ^’er  aod  transmftted  to  the  coftftcctir^ 
fdcaive'.  Another  poss.'h  tty  ^oulo  be  to  «se  a nog'i?  set  of 
teMp’atrts  «<h‘':h  has  be*  ^ averaged  ever  many  speas^rs 
However,  tower  outhly  synthesis  can  rve  expected  -vith  this 
method 

fn  odditrOn  to  the  obwtotis  redjctfOft  US  bit  rate,  the*a 
ere  other  advantages  te  t^e  use  of  t^se  techniques.  At 
first,  the  addit:bna<  processing  reeded  io  vegr^nt  and 
claisity  syee-cn  would  seem  to  reiu*t  »n  fiovier  vocoder 
operation,  howev  er  this 's  not  the  case.  Crwce  (he  tegmepfs 
art  krvowrs,  the  time  consjmtfvg  igtckorrelation  analysis  /teed 
be  pr efu'  n^ed  oniy  once  pc?  se^.'^eri*..  Th^i.  overaff  -ococer 
operation  It  actca-'y  fister.  Pur(hei»*orc.  s*n:e  gross 
aegmenl  class<f<ations  t.'-e  obtained  curing  the  segmeniaiion 
process,  specialized  processing,  depending  Oi  in*  segment 
Class  cart  be  ^refcrmed  Ter  t^gtri^it,  sifencts  can  be 
dismissed  with  no  processing,  a^d  5iw  coefficient  LPC 
analysts  can  be  preformed  for  fr.vafivesu  This  shoskd  result 
in  a more  accurate  synthesis. 

The  mfin  pgmt  should  be  (iear:  Iruougn  the  use  of 
apeciViged  Vr.ow‘edga  of  t^e  nature  of  speech,  and  higher 
kvel  gfgnai-iO-symbOf  Iransfor  matron  tecn-'dues, 
incrementally  better  vocoders  can  be  obtared.  We  ha«e 
dernonstf ated  two  steps  in  tt>$  progression.  The  first  vas 
the  transition  from  systems  baseo  so*eiy  on  spr;*traf 
analysis,  to  a system  (hat  corvtmed  Knowledge  ot  segmer^ts 
with  tpeclt^al  analysis  The  next  step  was  the  use  ot 
ladfti.ng  in  addition  10  S^me-'fatiOn  to  P.'va  even  further 
bandwidth  rcdwCfiOiL  As  sneecn  .'ecog*‘'.ticn  s/stems  ev.'ve, 
batter  and  better  encoongs  wM  become  practic^t. 
Eventuaify,  it  should  be  pcss.b<s  transmit  s/liaba  v;ed 
units. 

Finaliy,  improvensent  m coding  e»f  cienc>-  is  obtained 
•t  the  expense  of  generality  As  r.i.sre  sDcc>ai>zcd 
krcwlei*33  of  -ocec^i  ard  language  -c  used,  t.he  var  tr?>  of 
sounds  that  can  be  (ransmiirea  *5  '■eouced.  At  !ne  iowesi 
level  IS  the  system  that  transmits  ia.’*c5cd  speech  directly. 
With  (hs  system,  arpilri'y  sounos  c<n  be  repre  ented 
eccuratefy.  The  step  to  conve'^fiOra*  vococers  hmits  those 
founds  which  ca''  be  •ransmittes  ;o  speech.  Greater 
rcatnctions  occur  as  the  vocoder  necomes  more  ard  mp/e 
language  oriented. 

COCLvSICf.6 

We  have  presented  two  techniQues,  based  on 
algorithms  devtioped  t^e  Heais*/  and  narpy  speech 

recognition  systems,  v.‘'.:.h  use  K%s-w-'»cee  about  sOvOCh 
phenomena,  to  yield  reauc.f.ons  in  vccpcer  oamjw.O^h.  Vibnle 
the  degree  of  •mprovement  *ar-es  from  system  ic  lysiem. 


1>0C^'.  'siKltOft.  (ac:o'-;  -ang-ng  tre^i  3 lo  4 can  It., 
• ipicc'pd  from  eath  tv^if^d.  fev/hermore,  leiproverwyn-fi  by 
a factor  c?  5 or  more  cat>  be  retkiped  f the  Cachr^^yes  are 
COtnblned. 

^ segr'^enfai.bn  caused  no  noi-cetble  degrada{>pn 
in  ,‘he  syrnfSetic  sprec’i  ?;.:»>  ry  Wlh  iaoe^.^ng,  conr^dtiabio 
dfgradalion  occured,  rkp«.ever  i1  is  felt  that  ths  can  b3 
eliminated  wi^h  better  lenip^a!c’« 

Some  of  ti^e  pr  Kticel  aspects  of  vocoder 
iM^id mentation  bated  on  these  teervuoues,  atong  with  the 

lovpnta^e)  anp  dsa^v.^?ni»^e*  to  trve  us/?  of  ipec^afiaed 
knowledge,  were  dticussjd  On  the  bMri  ot  argimwrrtt 
presenieci  fhe^  we  be'.-e-e  lhai  speech  ana./sis-fyhthes»5 
tiSiis^  scgrrerttafrp.n  and  labeling  »s  worthy  of  (urthief 
reseerctv 
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ABSTRACT 


Many  combinatorial  search  probletns  can  be  u'.evren  within  the  "Chinese  restaurant 
menu  selection  paradigm  ' of  "choose  one  from  Column  A,  one  from  Column  B,  A solution  Ic 

such  a problem  consists  of  a set  of  selections  whicn  are  mmually  consistent  according  to  some 
set  of  consiralnis  The  onera'l  value  of  a solution  Is  a composite  function  of  the  value  of  each 
Individual  selecl'on  The  goal  of  the  search  Is  to  find  Ihi  best  (highest-rated)  solution. 
Examples  of  such  search  problems  occur  In  tfic  domains  of  speech  iinderslbnding,  vision,  and 
.■n  nlcal  diagnosis. 

This  pacer  describes  a scarch-prunirg  heuristic  and  a hailing  condition  which  are 
lot'.servatlve  in  that  they  will  not  ml.s.  the  best  solution  by  pruning  11  out  of  the  search  or  by 
terminating  Ihe  search  brfore  !;  Is  footid.  The  method  exploits  Information  aboul  already- 
found  soiMiions  In  order  to  prune  ihe  searen  and  decide  when  to  terminate  It.  An 
Implemcntalloh  of  rhs  halting  condlMon  and  pruning  heuristic  within  the  Heaisay-ll  speech 
underctanding  system  is  described  and  evaluated,  and  the  conditions  governing  Its 
applUaPUUy  and  performan'e  are  di.sccssed. 

ti^nBOOVCTIOS':  SOr/IE  EX/lV/Pl.e* 

Ai  frcquertly-occvirlrig  problem  In  Al  Involves  'tndlng  the  best  combination  of 
“holcts  .'or  a s<*t  of  Interdependent  m vlilple- c holce  decisions  The  possible  combinations  form  a 
combinatorial  search  space  Each  decisKni  corresponds  to  a data  element  which  can  be  labelled 
(explained,  Inlerorctedf  In  several  alternative  ways,  some  of  which  may  be  prel'srable  to  (more 
appropriate  than)  others  Legal  sclutlons  f combinations  ol  labels)  must  satisfy  certain  domtln- 
spv'.lllc  consistency  constraints  governing  'hr  In'erdeper.denclei  betweon  the  various  elements 
to  be  l.nbelled 

One  example  of  comblnalctial  search  occurs  l.i  the  domain  of  speech  understanding. 
A spokert  ul.eiance  can  be  viewed  as  a set  of  contiguous  points  In  time  The  comblnaiorlal 
serreh  task  of  a speecli  understanding  system  Is  to  label  each  tlmo  Interval  with  the  word 
apparently  spoken  dur.ng  that  l.nierval  Several  labels  may  appear  plausible  due  to  the 
uncertainly  or  the  speech  "Ignal  and  the  rd  recognition  process  [7],  A solution 
consists  of  a Ira-iscriptloo  i f the  utterance,  !.e.,  a seouence  of  word  labels,  which  Is 
sy ntactlcai'y  and  semantically  consistent.  The  v'edibll.ty  (prcbatilllty  of  correctness)  of  su-.h 
a solution  depends  or.  tne  overall  goodness  ol  r|t  between  the  labels  ;•  nd  their  time  Intervals 

Another  example  comes  from  "he  domain  of  vision.  The  -ontoiir  detection 


1 This  work  was  supporlcd  In  part  by  the  Def-  nse  Advanced  Research  Projects  Agency  under 
contract  no  Fd462C-73-C-0074  and  —onllored  by  the  Air  Force  Office  of  Scientific 
Research.  In  addition,  the  author  was  partially  supported  by  a National  Science  Foundation 
Graduate  Fellowship,. 
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problem  can  be  described  as  follows:  given  a scene  represented  by  an  array  of  pixel  gray 
levels,  label  each  pixel  with  a vector  corresponding  to  the  apparent  Intensity  gradient  at  that 
point  In  the  Image  [9],  A consistent  Inlerpreiatlon  of  the  scene  assigns  parallel  gradients  to 
contiguous  pixels  on  a contour  and  null  gradients  to  pixels  In  the  Interior  of  a region.  The 
accuracy  of  an  Interpretation  depends  on  the  overall  degree  to  which  the  labels  match  the 
visual  data  they  attempt  to  describe. 

A third  example  can  be  found  In  the  domain  of  medical  diagnosis.  Here  the  data 
elements  to  be  explained  are  (he  patient's  symptoms.  A diagnosis  provides  consistent 
explan -stlons  for  all  the  symptoms.  The  plausibility  of  a diagnosis  depends  on  the  overall 
plausibility  with  which  the  Individual  symptoms  are  accounted  for  [l]. 

PROPERTIES  OF  COMBINATORIAL  SEARCH 

Let  us  now  examine  these  search  problems  In  order  lo  discover  common  properties 
whlc:  can  be  exploited  In  designing  hatting  condlllons  and  pruning  heuristics.  In  each 
example,  the  set  of  data  elements  (points  In  time,  pixels,  symptoms)  to  be  explained  or  labelled 
Is  known  .St  the  beginning  of  the  search.  (Actually,  this  assumption  does  not  hold  for  systems 
like  MYCilJ  which  collect  data  during  the  course  of  the  search.  However,  as  we  shall  see.  It  Is 
sufficient  for  the  set  of  elements  to  be  determined  anytime  before  the  first  solution  Is 
found. ) 


.A  partial  solution  consists  of  consistent  explanations  for  a subset  of  the  elements. 
Ci'jmbinstorlal  search  algorithms  typically  extend  and  combine  such  partial  solutions.  In 
fact,  each  step  In  the  search  can  be  characlerl.-.ed  as  examining  a collection  of  partial 

.solutions  1 1 Ij^,  and  then  possibly  creating  a new  p.irtla!  solution  I’  We  can  use  rating 

Infotnaiion  about  partial  solutions  in  order  to  decide  when  lo  halt  the  search  once  some 
solution  has  been  found.  For  example,  suppose  we  examine  the  ratings  of  all  existing  partial 
solutions  and  conclude  that  none  of  them  can  be  extended  Into  a complete  solution  rated 
higher  than  the  be.st  one  found  so  far.  Under  this  condition,  It  Is  safe  lo  halt  the  search;  the 
best  solution  found  Is  the  best  one  possible.  This  condition  Is  the  desired  conservative 
hailing  condition. 

A similar  technique  can  be  used  lo  prune  the  search.  If  a partial  solution  cannot 
possibly  be  ex'rapolated  l.ntc  a complete  solution  superior  lo  the  best  existing  one,  It  can  be 
rejected  --  I e , all  efforts  lo  extend  It  or  combine  It  with  other  partial  .solutions  can  safely  be 
abandoned.  This  pruning  neurlsilc  Is  conservative  but  also  rather  weak.  A more  powerful 
heuristic  depends  on  certain  properties  of  the  function  used  for  rating  solutions.  Let  us 
consider  this  funct.on  In  mote  detail. 

THE  RATING  fCVCTfO.V 

A complete  solutlor'  ron.'si.Mently  explains  all  the  elements'  and  Is  rated  according 
lo  how  well  each  element  lsexplalr.ee.  l.e  . If  the  rallr.g  runcilon  B(I,S)  measures  how  well 
the  Interpretation  I explains  the  elements  of  the  set  S,  then  R(I,S)  = f(P,(l,e)  | e In  S).  where 
R(l,e)  measures  how  well  I explains  l';e  element  e.  B(I,.S)  Is  assumed  to  be  an  Increasing 
function  of  the  terms  R(l,e)  The  Inlerpieullon  I Is  a set  of  labels  for  the  elements  of  S,  l.e  , 
for  all  e In  S l:e  ->  1|(  e>.  The  rating  Pd.e)  may  be  context  - sensltl  ve,  I e , depend  on  how  other 
elements  besides  e .ere  labelled  (e.g  , Its  neighbors,  if  e is  a pixel).  A considerable 


1 This  condition  could  be  'elaved  by  , ilc'.ving  comp'ete  solutions  lo  label  some  elements 
ICiNORED  " The  rating  tun  .Mon  would  then  have  lo  reflect  the  relative  significance  of 
explaining  or  Ignoring  a .er.  clemrm,  so  as  lo  allow  meaningful  comparison  between 
solutions  accounll  ig  for  tlirferenl  subsets  of  the  element  set. 


simplification  Is  possible  If  B(l.e)  Is  context-free,  I e.,  R(I,e)  • R(l](e),e),  where  lj(e)  t<  the 
label  assigned  by  I to  e,  and  R(l,e)  measures  the  goodness  of  fit  between  the  label  I and  the 
element  e.  In  this  case,  R(1,S)  • f(R(l,(e),e)  | e In  S>.  If  f Is  a simple  averaging  function, 
then  R(I,S)  “ Average  {R(l|(e),e)  I e In  S) 

The  best  solution  I maximizes  R(1,S)  subject  to  the  cons'stency  constraints.  Note 
that  the  function  R may  produce  higher  values  If  applied  to  Inconsistent  Interpretations 
(non-solutions).  For  example,  the  Interpretation  lmax:e  ->  Imax(e).  where  Imax(e)  Is  the 
highest-rated  label  for  e,  will  In  general  maximize  R(I,S>  bulls  not  In  general  consistent. 

A HALTING  CONDITION  AND  PRUNING  HEURISTIC 

We  cap  now  precisely  define  our  halting  condition  and  pruning  heuristic  in 
terms  of  the  ratln/,  unction  R.  Let  S'  be  a subset  of  the  element  set  S,  and  let  1'  be  a partial 
solution  which  explains  S'.  Let  I be  the  highest-rated  solution  found  so  far  during  the 
search. 

r can  be  ext^  .ided  Into  a complete  (not  nere.csarlly  consLstent!)  Interpretation  I" 
by  assigning  Imax(e)  to  every  e In  S-S'.  I"  is  the  highest-rated  possible  complete  extrapolation 
of  r.  Thus  ;f  R(r',S)  < R(I,S),  r cannot  be  extended  Into  a solution  better  than  I,  and  It  Is  safe 
to  reject  I'  and  all  Us  potential  extensions  Unfortunately,  this  condition  Is  too  strong 
and  Is  not  often  satisfied.  A more  powerful  (but  still  conservative)  pruning  he’jrlstic  Is 
made  possible  by  assuming  that  B is  context-free  In  the  sense  defined  earlier. 

A MORE  POWERFUL  PRUNING  HEURISTIC 

Suppose  that  R Is  context-free  and  that  a solution  I has  been  found.  If  a better 
solution  Is  possible,  there  must  exist  a partial  soluUon  I'  which  Is  locally  superior  to  I.  1’  Is 
locally  superior  to  I over  domain  S'  If  R(r,S’)  > R(I,S').  Intuitively,  I'  explains  some  subset  S' 
better  than  I does.  If  no  such  I'  exists,  then  I Is  the  best  solution,  and  It  Is  safe  to  halt  the 
search 


This  reasoning  requires  some  Jusilf Icaiion.  We  consider  all  Individual  element 
labels  to  be  one-element  partial  solutions,  and  assume  that  they  are  available  to  the  search 
algorithm  as  such.  If  some  potential  complete  solution  I " Is  bettei  than  1,  then  there  must  exist 
at  least  one  element  e In  S such  that  R(r’,e)  ' R(l|,.(e),c)  > B(l|(e),c)  ^ R(l,e).  (Otherwise 
Rd  '.S)  < B(l,5)  ) This  one-element  partial  solution  can  be  extended  step  by  step  Into  1"  so  that 
the  partial  solution  I'  at  each  step  Is  locally  superior  to  I.  We  assume  that  such  a sequence  of 
partial  solutions  can  be  found  by  the  search  algorithm  This  Is  a strong  assumption.  Many 
sequences  of  partial  solutions  may  lead  by  stepwise  extension  and  combination  to  the  same 
solution,  but  not  all  will  maintain  local  superiority  at  each  step,  and  not  all  may  be  realizable 
by  the  search  algorithm  being  used 

With  this  caveat,  we  now  observe  a happy  property  of  context-free  rating 
functions:  once  a solution  has  been  found,  only  partial  solutions  which  are  locally 
superior  to  It  need  be  considered.  All  others  may  be  deactivated,  I e..  Ignored  except  for 
combination  with  active  partial  solutions. 

We  can  now  express  a powerful  conservative  pruning  condition:  A proposed  search 
operation  based  on  partial  solutions  Ij,  , Ij^  may  safely  be  cancelled  If 

( I ) Any  of  the  I|  has  been  rejected,  or 

(2)  All  of  the  l|  have  been  deactivated. 
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The  hailing  condition  Is  trivial;  halt  when  all  pending  search  operations  have  been 


cancelled. 

UNDEHLYING  ASSUMPTIONS 

Let  us  now  re-examine  some  of  the  assumptions  on  which  this  method  Is  based,  and 
the  motivations  for  making  them. 

(1)  The  rating  function  Is  context-free.  Otherwise  the  local  superiority  criterion  Is 

not  valid. 

(2)  The  labels  Imax(e)  are  known  at  the  beginning  of  the  search,  and  exist  as 
one-point  partial  solutions.  Othervrise  correct  but  low-rated  partial  solutions  might  be 
erroneously  rejected.  Actually.  In  order  tc  avoid  erroneous  rejection,  It  Is  only  necessary  to 
know  an  upper  bound  function  Rmax(e)  > R(l.e)  for  all  e In  S.  The  tighter  this  upper  bound, 
the  more  partial  solutions  can  be  rejected.  The  Bmax  function  used  by  the  HV^IM  speech 
understanding  system  Is  defined  by  the  score  of  the  best  phonetic  label  for  each  segment  [8]. 
Since  this  score  Is  based  on  the  best  possible  word  match  for  each  segment  rather  than  on  the 
best  actual  word  match.  It  provides  a poor  (over-opllnlstlc)  upper  bound  on  the  actual  word 
ratings,  and  produces  mediocre  results  The  Rmax  function  used  In  Hearsay-II  la  defined  by  the 
score  of  the  highest- rated  hypothesized  word  at  each  point  In  the  utterance,  and  produces  good 
results. 

(3)  If  a potenllal  solution  I"  Is  belter  than  an  existing  solullon  I,  the  search 
algorithm  must  be  capable  of  building  I"  In  such  a v/ay  that  each  partial  solution  1'  In  the 
derivation  sequence  Is  locally  superior  to  1.  Otherwise  the  derivation  of  1"  might  require 
operating  on  a set  of  deactivated  partial  solutions  and  be  blocked  by  the  deactivation  pruning 
heuristic. 

EXAMPLE  THOM  HE  APS  AY -11 

The  Hearsay-ll  speech  undersiandlng  system  [2]  segments  a spoken  utterance  Into 
syllable- length  time  Intervals.  The.se  are  Ihe  elements.  The  labels  for  each  element  are  taken 
from  a 1,000-word  vocabulary.  A complete  solullon  Is  a grammatical  lianscriptlon  spanning 
Ihe  utterance.  A partial  solullon  Is  a grammatical  phrase  spanning  part  of  Ihe  utterance. 
The  rating  function  Is  a simple  average  of  label  fll  goodness.  A (partial)  solution  I covers  a 
lime  Interval  [flrst'sylTasl'sylJ.  lls  rating  Is  Its  average  word  rating  weighted  by  the  number  of 
syllables  In  each  word.  I.e.,  R(l,[flrsrsyMast’syl])  = Average  ( R(W|(syl)  ! A(syl)))  ),  where 
firsl'syl  < syl  < last'syl,  A(syl)  represents  the  acoushc  data  In  the  interval  syl,  W|(5yl)  Is  the 
word  label  assigned  by  I to  syl,  and  R(W  | A)  measures  how  closely  Ihe  word  W matches  the 
acousfc  data  A.  R(W  | A)  Is  computed  by  the  word  verifier  [6]. 

In  Hearsay-ll,  partial  solutions  are  explicitly  represented  as  hypotheses  on  a global 
data  structure  called  a blackboard.  Search  operations  are  proposed  by  various  knowledge 
sources  which  monitor  the  data  or.  the  blackboard  The  operations  relevant  to  the  discussion  at 
hand  are  [5] 


(1)  Pecognitlon  given  a sequence  of  words,  parse  It  and  record  It  as  a partial 
solullon  If  It  Is  grammatical. 


(2)  Prediction.  given  a recognized 
g r a nr  irr  a l i c ai  i y precvde  or  follow  lx.  Ptcdictlc.r.s  -whlcl 
the  word- verifier  are  recorded  on  the  blackboard  as 


phrase,  propose  words  which  can 
i arc  rated  above  a specified  threshold  by 
one-word  hypotheses.  Thus  prediction 


dynamically  assigns  extra  labels  to  elements,  and  could  potenllally  violate  our  earlier 
assumption  that  Rmax(e)  Is  known  before  the  rejection  pruning  heuristic  Is  applied.  This  Is 
not  a problem  In  practice,  hovA/ever,  since  most  label  assignment  (word  recognition)  is  done 
at  the  beginning  of  the  search  cr  before  the  first  complete  solution  Is  found,  and  predicted 
words  are  aeldom  rated  higher  than  the  best  pre  'iously-recognlzed  words. 

(3)  Concatenation:  given  two  temporally  adjacent  phrases  (or  a phrase  and  a 
word  predicted  next  to  It  and  subsequently  verified),  concatenate  them  and  record  the 
result  as  a partial  solution  If  it  is  grammallcal. 

These  search  operations  are  performed  In  order  of  their  priorities,  which  are 
assigned  by  a central  focus-of-aitentlon  module  [3].  The  focus  module  tries  to  order  the 
search  In  a best-first  manner,  and  succeeds  about  50V.  of  the  time  on  the  corpus  tested  for 
this  paper.  This  figure  seems  to  Increase  as  the  constraints  on  grammatical  consistency  are 
Increased,  I.e.,  as  the  branching  factor  of  the  language  Is  reduced.  For  a best-first  search,  the 
best  halting  policy  Is  to  terminate  the  search  as  soon  as  a solution  Is  found.  Note  that  the 
rejection  and  deactivation  pruning  heuristics  are  Inapplicable  If  this  policy  Is  used,  since 
these  heuristics  do  not  become  applicable  until  some  solution  Is  found. 

CVAtUATtOS 


The  deactivation  and  rejection  heuristics  were  evaluated  on  a corpus  of  34  utterances 
drawn  from  a 262-word  vocabulary.  Utterance  length  ranges  from  3 to  9 words,  with  an 
average  of  6 The  fanout  (number  of  grammatical  word  successors  In  each  word  position) 
averages  27  for  the  corpus. 

Each  utterance  was  processed  In  6 modes.  Mode  N uses  neither  heuristic;  mode  R 
u.ses  rejection;  mode  0 u.ses  deactivation;  and  mode  B uses  both.  Ir  mode  F,  the  system  accepts 
the  first  solution  It  finds  and  Immediately  halts.  The  results  of  the  experiment  are  shown  In 
Table  1. 


The  simple  accept-the-flrsl-solullon  policy  u.sed  In  mode  F Is  fastest,  but  at  a 
considerable  cost  In  accuracy,  since  It  falls  for  those  runs  (about  50'/.)  In  which  the  highest- 
rated  solution  Is  not  the  first  one  found.  A more  conservative  policy  finds  these  solutions  at 
the  cost  of  extia  search  In  those  runs  where  the  best  solution  Is  found  first.  The  correct 
choice  of  policy  (simple  versus  conservative)  depends  on  a tradeoff  between  efficiency  and 
accuracy.  Since  accuracy  Is  very  Important  In  speech  understanding,  the  conservative 
policy  Is  preferred  despite  Its  extra  cost. 

The  heuristics  can  be  evaluated  according  to  two  criteria.  First,  how  fast  Is  the  best 
solution  found  once  the  first  solution  Is  found’  As  Table  1 shows,  deactivation  Is  about  twice  as 
powerful  as  rejection  In  speeding  up  this  phase  of  the  search.  The  combination  of  heuristics  Is 
only  slightly  more  effective  than  using  deactivation  alone. 
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Mode;  N R D B F 

Average  number  of  search  operations  (Hearsay-ll  knowledge  source  and  piecondition 
eKecullons)  to  find  first  solution;* 

15i  157  145  152  162 

Average  number  of  (percent)  extra  search  operations  to  iind  the  best  solution  once 
the  first  solution  has  been  found: 


71 

58 

30 

26 

0 

46% 

37% 

21% 

17% 

0% 

Speedup  In  this  phase  of  the  search  relative  to  mode  N; 

1 1.2  2 4 2.7  Infinity 

Average  total  number  of  search  operations  lo  find  best  solution: 

223  215  176  178  152 

Average  number  of  (percent)  extra  search  operations  to  satisfy  halting  condition^ 

' ~ 3 

once  the  best  solution  has  been  found  (excluding  runs  Ir*  which  time  or  space  Is  exhausted;-. 


241 

153 

89 

52 

0 

108% 

7J% 

61% 

29% 

0% 

Average  total  number  of  search  operations  until  halting  condition  Is  satisfied; 

286  282  263  226  162 

Number  (percent)  of  utterances  In  which  ha'ting  condition  Is  satisfied  before  system 
exceeds  predefined  limits  on  lime  (800  search  operations)  or  space  (193K); 


4 

17 

32 

32 

34 

12% 

50% 

94% 

94  V. 

1 00% 

Table  1.  Results  of  experimental  evaluation  of  pruning  heurlstlca. 


1 Ideally  these  numbers  should  be  equal,  since  the  heuristics  are  not  applied  until  the  first 
solution  Is  found  The  variation  l.n  these  figures  Is  caused  by  some  randomness  In  the 
Hearsay-ll  scheauler  In  choosing  between  equally  promising  search  operations. 

2 The  halting  condition  Is  satisfied  when  no  more  search  operations  are  pending,  or  when  all 
the  pending  operations  are  considered  unpromising  by  the  system. 

3 Speedup  ratios  between  different  modes  are  not  meaningful  here  since  the  set  of  excluded 
utterances  varies  from  mode  to  mcde. 
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Second,  how  fjjt  U the  halting  condition  satisfied  once  the  best  solution  la  found’’ 
An  Ideal  policy  would  halt  as  soon  as  the  best  solution  was  found.  The  deviation  of  an  actual 
policy  from  this  Ideal  can  be  measured  by  Its  “halting  overhead,”  I.e.,  the  amount  of  extra  search 
performed  after  the  best  solution  is  found.  When  neither  heuristic  Is  used,  the  halting  condition 
Is  satisfied  In  only  12V.  of  the  runs  (time  or  space  bounds  are  exceeded  In  the  others)  and  the 
halting  overhead  In  those  runs  Is  108V..  The  rejection  heuristic  succeeds  In  satisfying  the 
halting  condition  In  501C  of  the  runs,  with  an  overhead  of  7lX.  Deactivation  leads  to  halting  In 
94V.  of  the  runs,  with  SIV.  overhead.  The  combination  cf  both  heu.lstics  also  causes  halting  In 
94V.  of  the  runs,  but  reduces  overhead  to  only  29V.. 

These  results  can  be  summarized  as  follows: 

(1)  Deactivation  Is  about  twice  as  powerful  as  rejection  In  accelerating  the  search 
for  the  best  solution  once  the  first  solution  has  been  found.  This  difference  in  empirical 
performance  substantiates  the  Intuitive  notion  that  the  conditions  for  deactivating  a partial 
solution  are  substantially  easier  to  satisfy  than  the  conditions  for  rejecting  It.  The  combined 
heuristics  speed  up  this  phase  of  the  search  by  a significant  factor  (2.7). 

(2)  The  combined  heuristics  succeed  most  (94V.)  of  the  time  In  satisfying  the  halting 
condition,  at  a reasonable  cost  (29V.)  compared  to  the  time  It  takes  to  find  the  best  solution.  The 
large  variance  In  this  cost  and  the  failure  to  satisfy  the  halting  condition  In  the  other  6%  of  the 
runs  suggest  that  other  techniques  are  needed  to  further  streamline  the  aearch  without 
eliminating  the  best  solution. 

07SCUSS/01V  or  APPUCABILITY 

What  properties  of  Hearsay-ll  make  this  method  applicable’ 

(1)  Most  of  the  word  labelling  is  performed  before  the  first  solution  Is  found  and 
the  heuristics  are  applied.  Seldom  Is  a new  word  subsequently  hypothesized  with  a rating 
higher  than  all  the  other  words  In  Its  time  Interval  Thus  the  necessary  Information  (the 
Bmax  function)  Is  determined  before  the  heuristics  are  applied  Exceptions  do  not  automatically 
cause  erroneous  rejection,  since  the  Pmax  function  generally  provides  a safety  margin  by 
overestimating  the  rating  of  the  best  possible  solution. 

(2)  A solution  must  account  for  the  whole  lime  Interval  of  the  utterance,  I e , for 
every  element  (syllable).  This  facilitates  the  comparison  of  extrapolated  potential  solutions 
with  already-found  solutions 

(3)  The  rating  function  for  evaluating  solutions  Is  context-free.  This 

facilitates  the  local  comparison  of  partial  solutions  with  complete  solutions. 

The  context-free  property  !s  somewhat  counter-intuitive  since  the  consistency 
criteria  are  In  general  context  sensl'lve,  I.e..  the  admissibility  of  a label  depends  on  the 
labels  assigned  to  other  elements.  The  rating  function  might  be  expected  to  rate 
solutions  (consistent  Interpretations)  higher  than  Inconsistent  explanations,  but  a 
context-free  rating  function  does  not  have  this  Intuitively  satisfying  trait.  Our 
approach  separates  two  properties  of  a solution- 

(1)  satisfaction  of  consistency  constraints 


(2)  goodness  of  fll  between  labels  and  data. 


Consistency  Is  considered  to  be  an  all-or-none  properly  and  Is  guaranteed  by  the 
form  of  the  search.  Relative  goodness  of  fit  Is  assumed  to  be  locml,  rather  than 
context-sensitive.  When  this  assumption  approximates  the  truth,  It  becomes  possible  to  apply 
the  powerful  deactivation  heuristic. 

CONCUJSIOSS 

Conservative  pruning  heuristics  for  combinatorial  search  have  been 
presented  They  operate  by  eliminating  branches  of  the  search  which  cannot  lead  to 
solutions  belter  than  those  found  already.  In  this  respect,  they  can  be  thought  of  as 
alpha-beta  pruning  heuristics  In  a one-player  game.  The  pruning  heurlstlca  and  associated 
halting  condition  have  been  Implemented  In  Hearsay-11  and  shown  to  be  effective  In  *he  real- 
world  problem  domain  of  speech  understanding. 

When  the  object  of  a search  Is  to  find  the  best  solution  (not  Just  any  solution), 
there  is  an  Important  tradeoff  between  speed  and  accuracy.  The  simplest  halting  policy 
accepts  the  first  solution  found.  This  policy  Is  correct  If  the  search  Is  always  best-first;  the 
closer  the  search  Is  to  oest-first.  the  more  attractive  such  a simple  policy  becomes.  More 
sophisticated  policies  Increase  accuracy  at  the  expense  of  prolonging  the  search  so  as  to 
guarantee  that  the  fce.'t  solution  Is  not  missed. 

In  a nearly-best-flrst  search,  the  discovery  of  a solution  changes  the  purpose  of 
the  search  from  one  of  finding  the  best  possible  solution  to  one  of  verifying  that  there  Is  no 
better  solution  than  the  one  found.  This  change  of  purpose  should  be  reflected  In  the  search- 
guldtng  policies. 

The  approach  described  exploits  certain  assumptions  about  the  search. 

(1)  The  search  space  can  be  represented  by  a set  of  elements  (data)  each  of  which 
can  be  labelled  In  several  ways.  A solution  labels  all  the  elements  and  satisfies  specified 
consistency  constraints. 

(2)  A rating  function  evaluates  how  well  a given  label  fits  a given  element.  An 
upper  bound  on  the  best  label  rating  for  each  element  should  be  determined  by  Ih#  time  the 
first  solution  Is  found.  The  tighter  the  bound,  the  better  the  performance  of  the  pruning 
heuristics. 


(3)  The  rating  of  a solution  should  be  a function  of  the  ratings  of  Its  Individual 

labels  It  should  be  possihle  to  compute  an  upper  bound  on  the  rating  of  the  best  possible 

extrapolation  of  a given  partial  solution  The  tigh.er  ihe  bound,  the  better  the  performance. 

(4)  The  better  the  found  solution  relative  to  Ihe  best  (generally  Inconsistent) 
Interpretation  Imax  (which  assigns  each  element  Its  hlghesl-ianlted  label),  the  more  pruning 
tan  be  done  The  stronger  the  consistency  conslialnts,  ihi  lower  a solution  will  tend  to  be 
rated  compared  to  Imax,  and  the  worse  Ihe  performance. 

Many  search  problems  (e  g , speech  and  Image  understanding,  medical  diagnosis) 

appear  to  fit  the  paradigm  of  "choose  one  from  Column  A,  one  from  Column  B,"'  I.e.,  given 

alternative  choices  for  a set  of  decision  points,  find  the  best-rated  consistent  set  of  choices. 
Vtlhen  efficient  besl-ftrsl  search  algorithms  are  infeasible,  some  mechanism  Is  needed  for 
deciding  when  to  halt  the  search  and  accept  the  best  solution  found  »o  far.  Such  a mechanism 
should  terminaie  i'ne  seaicli  as  soon  as  possible  without  Ignoring  belter  solutions.  This  paper 
has  shown  how  such  a mechanism  can  exploit  Information  about  already -found 
solutions  to  accelerate  the  search  conservatively,  I.e  , without  Ignoring  better  solutions. 
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for  the  tr anscription  of  verbal  protocols  Behat/ioraL  Retearch  Methods  ar,d 
instrumentu-.ion  (May  1973). 

Civ..  3W2Civllj  Cof.iputer  Science  Speech  Group.  Working  papers  in  spt -ch  recognition 
I!.  Tr vn.  Report,  CMUCSD,  1973.  Includes  Bk73Ma,  Bm73Ne,  Er73Sy,  R373Mo 
and  ffe73Hi(. 


Er73Ro  Erma  L.  D.,  Lowerre,  B,  T.  and  Redd,,  D.  R.  Representation  and  use  of 

acoustic -phor.eiic  knowledge  in  the  Bearsay-l  system.  Sf'th  Meeting  Acous. 
Soc.  Amer.,  Eos  Angeles,  CA,  Ncv.,  1973,  49.  (Abstract  only) 

Er/3Rw  t rman,  1.  D.  and  Reddy,  0 R.  Report  of  a worksliop  on  machine  segmentation 
and  lot'Cling  o‘  con.iecled  speech.  Meeting  Acous.  Soc.  Amer.,  Los 

Angelos,  L.A,  Mo  - tbl  (Abstrari  only) 


Er73Sy  f r -van,  L P , F eni  .el  R,  0.,  Lesser,  V.  R and  Re  ddy,  D R.  System  organizations 
for  sueecn  .mdei  , tanding:  rnolic atiOi-.s  of  network  and  multiprocessor 
coi’pute'  dfvf-  ,:-(.tures  fo  Al.  Proc.  //CA.’  73,  St anf or d,  C A,  1973,  194-199. 
Also  appeared  m CM73VV?  xed  IEEE  Trans.  Comnatert,  C-25,  AiA.  4,  April, 

197(3,  J 14-421  Early  mohvatior.  lOr  the  Hearsav  ’.I  muiti-process  structure. 
fJe73Us  fieely,  R b Cn  '.lie,  use  o'  syita*  and  semantics  in  a speech  undersianding 
' ystt-m  Tech  Report,  CMLx. SC.  iPh.D  Dissertation,  S'anford 

ijniversitjp  Desc ' 'ptiQn  c:  tuy-e'd  processing  m Hearsay-1. 

"j*  73Sp  ! loweli.  / , aernett.  J , Fo  gie,  _■ , -Jrpt  n,  C , Kiatt,  D , '.icklider,  J.  C.  R.,  Munson, 
i , Reddy  . R and  Woods,  W.  1 l.'dersranding  Systems.  Final  Report  of  a 

Study  G- ?up  fvorth-Ho'lar.d,  1973  I jr.'i.n.-iiy  appeared  in  1971.  This  seminal 
work  set  Iho  goals  and  orientation  tor  ,he  i-RT’A  SDR  effort 


Re79Ey  Reddy,  D R Eves  a.mi  ears  for  comf  .lers.  i-'  ry  ' ..  C/  Eachtc gung  Cogntttva 
ver fahren  und  .lysteme  (Beckmann,  M , Goos,  C.  arp  r(  j-.'z‘  H.  P.,  Ed.) 

Spr  i,ig;r  - Ver  lag,  New  York,  NY,  l'j73,  1-2S.  Keynote  rJdre'.-, 'Tambu'g,  A^pril 

' 'J  / “ 
a -7  vu . 


Re73He  Reddy,  D,  R.,  Eiman,  L.  0.,  F'ennell,  R.  □,  Lowerre,  B.  T.  and  Neely,  R.  B The 
Hearsay  speech  understanding  system  86th  Meeting  Accus.  Soc.  Amer.,  Los 
Angeles,  CA,  Nov.,  1973,49.  (Abstra  I only) 

fte73Hf  Reddy,  D.  R.  The  Hearsay  system.  20  minute  16mm  sound  film.  Describes  the 
speech  understanding  problem  and  demonstrates  the  Hearsay-1  system. 

Prints  may  be  borrowed. 

Re73Hx  Reddy,  0.  R.,  Erman,  L.  D.,  Fennell,  R.  D.  and  f'leely,  R.  B.  The  Hears«y(-I) 

speech  understanding  system;  an  example  of  the  recognition  process.  Proc. 
ljCAI-73,  Stanford,  CA,  1973,  185-193.  Also  appeared  in  CM73W2  and  IEEE 
Trans.  Computers,  C-25,  No.  4,  April  1976,  422-431. 

Re73Mo  Reddy,  D.  R.,  Erman,  L.  D.  and  freely,  R.  B.  A mode!  and  a system  for  mactune 

recognition  of  speech.  IEEE  Trans,  on  Audio  and  Electroacoustics  AU~2l  {June 
1973)  229-238.  Also  appeared  in  CM73W2.  First  report  on  l-tearsay-1. 

Re73Pn  Reddy,  D.  R.  Phonemic  and  morphemic  variability  in  connected  speech.  66th 
Meeting  Aeous.  Soc.  Amer.,  Los  Angelos,  CA,  Nov.,  1973.  For  abstract  see 
JASA  55,  41 1. 

Re73So  Reddy,  D.  R.  Some  numerical  problems  in  artificial  intelligence:  implications  for 
complexity  and  machine  architecture.  In  CompleTtty  of  Sequential  and 
Parallel  Numerical  Algorithms.  (Traub,  J.  F.,  Ed.)  Academic  Press,  1973. 


- 1072  - 

CM72W1CMU  Computer  Science  “Speech  Group..  Working  papers  in  speech  recognitioa 
I.  Tech.  Report.  CMUC5D,  1972.  Includes  Er71In,  NE71Sp,  R870Sp,  Re70Cm, 
Re71Sp,  Re71Sm  and  Re72Me. 

Re72Me  Reddy,  D.  R.,  Erman,  L.  D.  and  Tteely,  R.  B.  A mechanistic  model  of  speech 
perception.  Proc.  1972  IEEE  Coaf.  Speech  Communication  and  Processing, 
Newton,  MA,  Apr.,  1972,  334-337.  Also  appeared  in  CM72W1.  (abstract  only) 


- 1971  - 

Er7llm  Erman,  L.  D and  Reddy,  D.  R.  Implications  ol  telephone  input  for  automatic 
speech  recognition.  Proc.  7th  Inter.  Congress  cn  Acoustics,  Vol.  3.  Budapest, 
Hungary,  1971,  85-88.  Also  appeared  m CM72W1.  Experiments  with  the 
Vicens-Reddy  system. 

Ne71Sp  Neely,  R.  B and  Reddy,  D.  R.  Speech  recognifio  ' in  the  presence  of  noise. 

Proc.  7th  Inter  Congress  on  Acoustics,  Vol.  3.  Budapest,  Hungary,  1971,  177- 
180.  Also  appeared  in  CM72WI.  Experiments  with  the  Vicens-Reddy  system. 
Re71Sm  Reddy,  D.  R , Bell,  C.  G.  and  Wulf,  W.  A Speech  recognition  in  a multiprocessor 
environment  Proc.  1971  IEEE  Conf.  on  Automatic  ControE  Miami,  Florida, 

1971.  Also  appeared  in  CM72W1. 

Re71Sp  Reddy,  0.  R.  Speech  recognition;  prospects  for  the  seventies.  Proc.  IF  IP 

1971,  Ljubljan?  Yugoslavia,  1971,  1.5  M3.  Also  appeared  in  CM72W1.  Invited 

paper. 


— 1970  - 


Gd70Am  Goodman,  G.  Ambiguity  in  phonetic  grammars.  Proc.  of  thm  3rd  Hau/aU  Int»r. 

Conf.  System  Sciences,  Honolulu,  Hawaii,  1970. 

Re70Cm  Reddy,  D.  R.,  Erman,  L.  D.  and  Neely,  R.  B.  The  CMU  speech  recognition 

project.  Proc.  1970  IEEE  System  Sciences  and  Cybernetics  Conf^  Pittsburgh, 
PA,  1970.  Also  appeared  in  CM72W1. 

Re70Sp  Reddy,  0.  R.  Speech  input  terminals  for  computers:  problems  and  prospects. 
IEEE  Computer  Conference,  Washington,  D.C.,  1970.  Also  appeared  in 
CM72W1. 

(The  following  earlier  papers  were  done  at  Stanford  and  are  included  here  for 
ref  erence.) 


- 1969  -- 

Ne69Ex  Neely,  R B.  Eypenmentol  conversational  computer  system.  J.  Acous.  Soc. 
Amer.  46  (July  1969)  89.  (Abstract  only) 

Re69Co  Reddy,  D.  R.  and  Neely,  R.  B.  Contextual  analysis  of  phonemes  of  English. 

Tech,  Report,  Stanford  University,  AI  Memo  79,  Stanford,  CA,  1969. 

Re69Se  Reddy,  D.  R.  Segment-synchronization  problem  in  speech  recognition.  J. 

Acous.  Soc.  Amer.  46  (July  1969)  89.  (Abstract  only) 

Re69Us  Reddy,  0.  R.  On  the  use  of  environmental,  syntactic,  and  probabilistic 

constiaints  in  vision  and  speech.  Tech.  Report,  Stanford  University,  Al  Memo 
78,  Stanford.  CA,  1969. 

Vi69As  Vicens,  P.  J.  Aspects  of  speech  recognition  by  computer.  Tech.  Report, 

Stanford  University,  Al  Memo  85,  Stanford,  CA,  1969.  Ph.D.  Dissertation.  The 
"Vicens-Reddy”  system. 

--  1968  - 

McGSCo  McCarthy,  J.,  Earnest,  L..  Reddy,  D.  R.  and  Vicens,  P.  J.  A computer  with  hands, 
eyes  and  ears.  Proc.  f JCC,  1968,  329-337. 

ReoSCn  Reddy,  D.  R.  Consonantal  clusters  and  connected  speech  recognition.  Proc. 

6tn  Inter.  Congress  on  Acoustics,  ToKvo,  .lapan,  1968,  57-60. 

Re68Co  Reddy,  D.  R On  computer  transcription  ol  phonemic  symbols.  J.  Acous.  Soc. 
Amer.  44  (Feb.  1968)  638-639. 

Re68Ph  Reddy,  D.  R.  and  Robinson,  A.  E.  Phoneme  to  grapheme  translation  of  English. 

IEEE  Trans.  Audio  and  Electroacoustics  16  (Feb.  1968)  2AO-246- 
Re6SPr  Reddy,  D.  R.  and  Vicens,  P.  J.  A procedure  for  segmentation  of  connected 
speech  J.  Audio  Engr.  Soc.  16  (Apr.  1968)  404-412. 

Vicens,  P.  Preprocessing  for  speech  ana  ysis  Tech.  Report,  Stanford 
Universiiy,  Al  Memo  71,  Stanford,  CA,  1968. 


:74 


Vi68Pr 


- 1967  -- 


Re67Co  Reddy,  D.  R.  Computer  recognition  of  connected  speech  J.  Acous.  Soc.  Amtr. 
(Aug.  1967)  329-347. 

Re67Ph  Reddy,  D.  R Phoneme  grouping  (or  speech  recognition.  J.  Acoia.  Soc.  Amer. 
41  (May  1967) 1295-1300. 

Re67Pi  Reddy,  D.  R,  Pitch  period  determination  of  speech  sounds.  Comm.  ACM  10 
(June  1967)  343-348. 


- 1966  - 

Re66Ap  Reddy,  D.  R An  approach  to  computer  speech  recognition  ty  direct  analysis 
of  the  speech  wave.  Tech  Report,  Stanford  University,  A1  Memo  43,  Stanford, 
CA,  1966.  Ph.D.  Dissertation 

Re66Se  Reddy,  0.  R.  Segmentation  of  speech  sounds.  J.  Acous.  Soc.  Amer.  40  (Aug. 
1966)  307-312. 


- 1964  - 

Re64Ey  Reddy,  0.  R.  Experiments  on  automatic  speech  recognition  by  a digital 

computer.  Tech.  Report,  Stanford  University,  A1  Memo  26  Stanford,  CA,  1964. 


Al>l'ir3viations: 

ASSP  --  Acoustics,  Speech  and  Signal  Processing. 

CMLCSD  --  Department  of  Computer  Science,  Carnegie-Mellon  University,  Pittsburgh, 
PA,  15213.  (412)  621-2600  X.  141. 

IJCAI  --  Inter  national  Joint  Coriferences  on  Artif'Cial  Intelligence. 

JASA  --  Journal  of  the  Acoustical  Society  of  America. 
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